Abstract :
[en] Deep learning has achieved remarkable results across a wide range of tasks, and last years have seen the emergence of large-language models that pushed artificial intelligence to its highest level. Yet its growing disconnection from the biological principles that originally inspired it has left several important challenges unaddressed: energy efficiency, long-term memory, continual learning, and the scalability of recurrent architectures. These challenges are remarkably well tackled by biological brains and neurons, through the use of specific properties. This motivates the field of neuro-inspired artificial intelligence that aims at abstracting and embedding biological mechanism inside artificial networks. This thesis investigates how two fundamental properties of biological neurons, i.e. their spiking activity and their bistability, can be translated into recurrent neural network architectures and training algorithms using dynamical systems theory, with the aim of addressing some of these limitations while retaining the practicality of modern deep learning.
Three contributions are presented, each grounded in dynamical systems theory. The first introduces the spiking recurrent cell (SRC), a recurrent layer whose internal dynamics are shaped using feedback theory to produce spiking activity that is closer to biology than classical neurons, while remaining trainable end-to-end. Deep networks built on SRC achieve competitive performance on neuromorphic benchmarks and exhibit robustness to noise. The second contribution establishes a quantitative link between multistability and long-term memory in networks through the variability amongst attractors (VAA) measure, and introduces the warmup, a pretraining algorithm that drives models towards multistable regimes. Warmed-up networks outperform their standard counterparts on long-term dependency tasks, and a double-layer architecture combining multistable and non-multistable units yields further gains. The third contribution resolves the tension between multistability and parallelizability by removing any transient dynamics from the model to solely focus on multistability. This gives rise to the concept of memory recurrent unit (MRU) and its concrete instantiation, the bistable memory recurrent unit (BMRU), that achieves strong performance on long-sequence benchmarks.
Taken together, these contributions demonstrate that biological mechanisms, carefully translated, can enrich recurrent neural networks with valuable properties. Validating these approaches on real-world tasks, scaling hardware implementations, and extending to other biological mechanisms such as dendritic processing or neuromodulation constitute natural directions for future work.
