mixed-feedback; spiking recurrent neural network; winner-take-all; Analog signal processing; Analogue filtering; Analogue signals; Mixed-feedback; Neural systems; Neural-networks; Neuromorphic hardwares; Spiking recurrent neural network; System use; Winner-take-all; Signal Processing; Computer Networks and Communications
Abstract :
[en] Neural systems use the same underlying computational substrate to carry out analog filtering and signal processing operations, as well as discrete symbol manipulation and digital computation. Inspired by the computational principles of canonical cortical microcircuits, we propose a framework for using recurrent spiking neural networks to seamlessly and robustly switch between analog signal processing and categorical and discrete computation. We provide theoretical analysis and practical neural network design tools to formally determine the conditions for inducing this switch. We demonstrate the robustness of this framework experimentally with hardware soft Winner-Take-All and mixed-feedback recurrent spiking neural networks, implemented by appropriately configuring the analog neuron and synapse circuits of a mixed-signal neuromorphic processor chip.
Disciplines :
Engineering, computing & technology: Multidisciplinary, general & others
Author, co-author :
Zendrikov, Dmitrii; Institute of Neuroinformatics, University of Zurich, Eth Zurich, Switzerland
Franci, Alessio ; Université de Liège - ULiège > Département d'électricité, électronique et informatique (Institut Montefiore) > Brain-Inspired Computing ; Wel Research Institute, Wavre, Belgium
Indiveri, Giacomo; Institute of Neuroinformatics, University of Zurich, Eth Zurich, Switzerland
Language :
English
Title :
Waves and Symbols in Neuromorphic Hardware: From Analog Signal Processing to Digital Computing on the Same Computational Substrate
Publication date :
2024
Event name :
2024 58th Asilomar Conference on Signals, Systems, and Computers
Event place :
Hybrid, Pacific Grove, Usa
Event date :
27-10-2024 => 30-10-2024
By request :
Yes
Audience :
International
Main work title :
Conference Record of the 58th Asilomar Conference on Signals, Systems and Computers, ACSSC 2024
Editor :
Matthews, Michael B.
Publisher :
Institute of Electrical and Electronics Engineers (IEEE)
This work was partially supported by the European Research Council (ERC) under the European Union's Horizon 2020 Research and Innovation Program Grant Agreement No. 724295 (NeuroAgents).
R. J. Douglas and K. A. Martin, "Recurrent neuronal circuits in the neocortex, " Current Biology, vol. 17, no. 13, R496-R500, 2007.
M. Carandini and D. J. Heeger, "Normalization as a canonical neural computation, " Nature Reviews Neuroscience, vol. 13, no. 1, pp. 51-62, 2012.
R. VanRullen and C. Koch, "Is perception discrete or continuous?" Trends in Cognitive Sciences, vol. 7, no. 5, pp. 207-213, May 2003.
X.-J. Wang, "Theory of the Multiregional Neocortex: Large-Scale Neural Dynamics and Distributed Cognition, " Annual Review of Neuroscience, vol. 45, no. 1, pp. 533-560, 2022.
V. Mante, D. Sussillo, K. V. Shenoy, and W. T. Newsome, "Contextdependent computation by recurrent dynamics in prefrontal cortex, " Nature, vol. 503, no. 7474, pp. 78-84, 2013.
L. Zeng, S. O. Skinner, C. Zong, et al., "Decision making at a subcellular level determines the outcome of bacteriophage infection, " Cell, vol. 141, no. 4, pp. 682-691, 2010.
I. D. Couzin, J. Krause, N. R. Franks, and S. A. Levin, "Effective leadership and decision-making in animal groups on the move, " Nature, vol. 433, no. 7025, pp. 513-516, 2005.
R. Sepulchre, G. Drion, and A. Franci, "Control across scales by positive and negative feedback, " Annual Review of Control, Robotics, and Autonomous Systems, vol. 2, no. 1, pp. 89-113, 2019.
N. E. Leonard, A. Bizyaeva, and A. Franci, "Fast and flexible multiagent decision-making, " Annual Review of Control, Robotics, and Autonomous Systems, vol. 7, 2024.
C. Mead, "Neuromorphic Engineering: In Memory of Misha Mahowald, " Neural Computation, vol. 35, pp. 343-383, 2023.
R. Hahnloser, R. Sarpeshkar, M. Mahowald, R. Douglas, and S. Seung, "Digital selection and analog amplification co-exist in an electronic circuit inspired by neocortex, " Nature, vol. 405, no. 6789, pp. 947-951, 2000.
B. B. Averbeck, P. E. Latham, and A. Pouget, "Neural correlations, population coding and computation, " Nature reviews. Neuroscience, vol. 7, no. 5, pp. 358-366, 2006.
A. Pouget, P. Dayan, and R. Zemel, "Information processing with population codes, " Nature Reviews Neuroscience, vol. 1, no. 2, pp. 125-132, 2000.
J. A. Bednar and S. P. Wilson, "Cortical maps, " The Neuroscientist, vol. 22, no. 6, pp. 604-617, Jul. 2016.
D. Zendrikov, S. Solinas, and G. Indiveri, "Brain-inspired methods for achieving robust computation in heterogeneous mixed-signal neuromorphic processing systems, " Neuromorphic Computing and Engineering, vol. 3, no. 3, p. 034 002, Jul. 2023.
U. Rutishauser, R. Douglas, and J. Slotine, "Collective stability of networks of winner-take-all circuits, " Neural Computation, vol. 23, no. 3, pp. 735-773, 2011.
U. Rutishauser, J.-J. Slotine, and R. Douglas, "Computation in dynamically bounded asymmetric systems, " PLOS Computational Biology, vol. 11, no. 1, O. Sporns, Ed., e1004039, Jan. 2015.
S. Moradi, N. Qiao, F. Stefanini, and G. Indiveri, "A scalable multicore architecture with heterogeneous memory structures for dynamic neuromorphic asynchronous processors (DYNAPs), " IEEE Transactions on Biomedical Circuits and Systems, vol. 12, no. 1, pp. 106-122, Feb. 2018.
P. Strata and R. Harvey, "Dale's principle, " Brain Research Bulletin, vol. 50, no. 5, pp. 349-350, 1999.
D. Angeli and E. D. Sontag, "Monotone control systems, " IEEE Transactions on automatic control, vol. 48, no. 10, pp. 1684-1698, 2003.
A. Bizyaeva, A. Franci, and N. E. Leonard, "Multi-topic belief formation through bifurcations over signed social networks, " arXiv preprint arXiv: 2308. 02755, accepted in IEEE Transaction on automatic control, 2025.
R. J. Douglas, C. Koch, M. Mahowald, K. A. C. Martin, and H. H. Suarez, "Recurrent excitation in neocortical circuits, " Science, vol. 269, no. 5226, pp. 981-985, 1995.
H. Jaeger, "From continuous dynamics to symbols, " in Dynamics, Synergetics, Autonomous Agents. World Scientific, May 1999, pp. 29-48.
H. Jaeger, B. Noheda, and W. G. van der Wiel, "Toward a formal theory for computing machines made out of whatever physics offers, " Nature Communications, vol. 14, no. 1, Aug. 2023.
A. Rubino, C. Livanelioglu, N. Qiao, M. Payvand, and G. Indiveri, "Ultra-low-power FDSOI neural circuits for extreme-edge neuromorphic intelligence, " IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 68, no. 1, pp. 45-56, 2020.
