Article (Scientific journals)
A bio-inspired bistable recurrent cell allows for long-lasting memory
Vecoven, Nicolas; Ernst, Damien; Drion, Guillaume
2021In PLoS ONE, 16 (6), p. 1-13
Peer Reviewed verified by ORBi
 

Files


Full Text
journal.pone.0252676.pdf
Publisher postprint (643.56 kB)
Download

All documents in ORBi are protected by a user license.

Send to



Details



Keywords :
recurrent neurol network; long-lasting memory; bi-stability; neuromodulation; biological plausibility
Abstract :
[en] Recurrent neural networks (RNNs) provide state-of-the-art performances in a wide variety of tasks that require memory. These performances can often be achieved thanks to gated recurrent cells such as gated recurrent units (GRU) and long short-term memory (LSTM). Standard gated cells share a layer internal state to store information at the network level, and long term memory is shaped by network-wide recurrent connection weights. Biological neurons on the other hand are capable of holding information at the cellular level for an arbitrary long amount of time through a process called bistability. Through bistability, cells can stabilize to different stable states depending on their own past state and inputs, which permits the durable storing of past information in neuron state. In this work, we take inspiration from biological neuron bistability to embed RNNs with long-lasting memory at the cellular level. This leads to the introduction of a new bistable biologically-inspired recurrent cell that is shown to strongly improves RNN performance on time-series which require very long memory, despite using only cellular connections (all recurrent connections are from neurons to themselves, i.e. a neuron state is not influenced by the state of other neurons). Furthermore, equipping this cell with recurrent neuromodulation permits to link them to standard GRU cells, taking a step towards the biological plausibility of GRU. With this link, this work paves the way for studying more complex and biologically plausible neuromodulation schemes as gating mechanisms in RNNs.
Disciplines :
Computer science
Computer science
Author, co-author :
Vecoven, Nicolas ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Systèmes et modélisation
Ernst, Damien  ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Smart grids
Drion, Guillaume ;  Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Systèmes et modélisation
Language :
English
Title :
A bio-inspired bistable recurrent cell allows for long-lasting memory
Publication date :
08 June 2021
Journal title :
PLoS ONE
eISSN :
1932-6203
Publisher :
Public Library of Science, United States - California
Volume :
16
Issue :
6
Pages :
1-13
Peer reviewed :
Peer Reviewed verified by ORBi
Available on ORBi :
since 15 June 2020

Statistics


Number of views
288 (31 by ULiège)
Number of downloads
174 (19 by ULiège)

Scopus citations®
 
9
Scopus citations®
without self-citations
8

Bibliography


Similar publications



Contact ORBi