Binding and multiple instantiation in a distributed network of spiking nodes

Sougné, Jacques

doi:10.1080/09540090110063507

Request a copy

Article (Scientific journals)

Binding and multiple instantiation in a distributed network of spiking nodes

Sougné, Jacques

2001 • In Connection Science, 13 (2), p. 99-126

Peer Reviewed verified by ORBi

Permalink
https://hdl.handle.net/2268/619

DOI
10.1080/09540090110063507

Files (1)Send to Details Statistics Bibliography Similar publications

Files

Full Text

Sougne2001ConnSci.pdf

Publisher postprint (1.14 MB)

Request a copy

All documents in ORBi are protected by a user license.

Send to

RIS BibTex APA Chicago Permalink X Linkedin

Details

Keywords :

Spiking Neural Network; Binding Problem

Abstract :

[en] An implementation of a distributed connectionist network of spiking neuron-like elements is presented. Spiking nodes fire at a precise moment and transmit their activation, with particular strenghts and delays, to nodes connected to them. Wen the potential of the node reaches a particular threshold, it emits a spike. Thereafter, the potential is reset to a resting value. The receiving nodes accumulate potential, but also, slowly lose their potential through decay. As with real neurons, after firing there is a short refractory period during which the node will be completely insensitive to incoming signals, after which its sensitivity will slowly increase. Precise timing properties are used to represent symbols in a distributed manner and also to solve the problems of variable binding and multiple instantiation. Several predictions about human short-term memory, predicate processing, complex reasoning and multiple instantiation arise from this model. This network shows how symbolic processing can be achieved using neurobiologically and psychologically plausible mechanisms that also have the advantages of generalization and noise tolerance found in connectionist networks.

Disciplines :

Neurosciences & behavior

Author, co-author :

Sougné, Jacques ; Université de Liège - ULiège > UDI FAPSE

Language :

English

Title :

Binding and multiple instantiation in a distributed network of spiking nodes

Publication date :

2001

Journal title :

Connection Science

ISSN :

0954-0091

eISSN :

1360-0494

Publisher :

Taylor & Francis Ltd, Abingdon, United Kingdom

Volume :

Issue :

Pages :

99-126

Peer reviewed :

Peer Reviewed verified by ORBi

Available on ORBi :

since 22 September 2008

Statistics

Number of views

84 (13 by ULiège)

Number of downloads

4 (4 by ULiège)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

Bibliography

Abeles, M., 1991, Corticonics: Neural Circuits of the Cerebral Cortex (New York: Cambridge University Press).
Abeles, M., Prut, Y., Bergman, H., Vaadia, E., and Aertsen, A., 1993, Integration, synchronicity and periodicity. In A. Aertsen (ed.) Brain Theory: Spatio-temporal Aspects of Brain Function (Amsterdam: Elsevier), pp. 149-181.
Agmon-Snir, H., Carr, C. E., and Rinzel, J., 1998, The role of dendrites in auditory coincidence detection. Nature, 393: 268-272.
Ajjanagadde, V., 1994, Unclear distinctions lead to unnecessary shortcomings: examining the rule vs fact, role vs filler, and type vs predicate distinctions from a connectionist representation and reasoning perspective. In Proceedings of the AAAI-94, Menlo Park, CA: AAAI Press, pp. 846-851.
Baddeley, A. D., and Warrington, E. K., 1970, Amnesia and the distinction between long and short-term memory. Journal of Verbal Learning and Verbal Behavior, 9: 176-189.
Baird, B., Troyer, T., and Eeckman, F., 2000, Attentional network streams of synchronized 40-Hz activity in a cortical architecture of coupled oscillatory associative memories. In D. S. Levine, V. R. Brown and T. Shirey (eds) Oscillations in Neural System (Mahwah, NJ: Lawrence Erlbaum), pp. 285-305.
Barlow, H. B., 1972, Single units and sensation: A neuron doctrine for perceptual psychology? Perception, 1: 371-94.
Barnden, J. A., and Srinivas, K., 1991, Encoding techniques for complex information structures in connectionist systems. Connection Science, 3: 269-315.
Bookman, L. A., and Alterman, R., 1991, Schema recognition for text understanding: an analog semantic feature approach. In J. A. Barnden and J. B. Pollack (eds) Advances in Connectionist and Neural Computation Theory Volume 1: High-level Connectionist Models (Norwood, NJ: Ablex Publishing Corporation), pp. 87-122.
Borisyuk, G., Borisyuk, R., Kazanovich, Y., and Strong, G., 2000, Oscillatory neural networks: modeling minding and attention by synchronization of neural activity. In D. S. Levine, V. R. Brown and T. Shirey (eds) Oscillations in Neural Systems (Mahwah, NJ: Lawrence Erlbaum), pp. 261-283.
Broadbent, D. E., 1975, The magic number seven after fifteen years. In A. Kennedy and A. Wilkes (eds) Studies in Long-term Memory (New York: John Wiley & Sons).
Brown, M. B., and Forsythe, A. B., 1974, The ANOVA and multiple comparisons for data with heterogeneous variances. Biometrics, 30: 719-724.
Canavier, C. C., Clark, J. W., and Byrne, J. H., 1990, Routes to chaos in a model of a bursting neuron, Biophysical Journal, 57: 1245-1251.
Cowan, N., 1998, Visual and auditory working memory capacity. Trends in Cognitive Sciences, 2: 77-78.
Cowan, N., in press, The magical number 4 in short-term memory: a reconsideration of mental storage capacity. Behavioural and Brain Sciences, 24.
Dyer, M. G., 1991, Symbolic neuroengineering for natural language processing: a multilevel research approach. In J. A. Barnden and J. B. Pollack (eds) Advances in Connectionist and Neural Computation Theory Volume 1: High-level Connectionist Models (Norwood, NJ: Ablex Publishing Corporation), pp. 32-86.
Eckhorn, R., Bauer, R., Jorden, W., Brosch, M., Kruse, W., Munk, M., and Reitboeck, H. J., 1988. Coherent oscillations: A mechanism for feature linking in the visual cortex? Biological Cybernetics, 60: 121-130.
Engel, A. K., König, P., Gray, C. M., and Singer, W., 1990, Stimulus-dependent neuronal oscillations in cat visual cortex: inter-columnar interaction as determined by cross-correlation analysis. European Journal of Neuroscience, 2: 588-606.
Engel, A. K., König, P., Kreiter, A. K., and Singer, W., 1991a, Interhemispheric synchronization of oscillatory neuronal responses in cat visual cortex. Science, 252: 1177-1179.
Engel, A. K., König, P., and Singer, W., 1991b, Direct physiological evidence for scene segmentation by temporal coding. Proceedings of the National Academy of Sciences, USA, 88: 9136-9140.
Engel, A. K., Kreiter, A. K., König, P., and Singer, W., 1991c, Synchronisation of oscillatory neuronal responses between striate and extrastriate visual cortical areas of the cat. Proceedings of the National Academy of Sciences, USA, 88: 6048-6052.
Feldman, J. A., 1982, Dynamic connections in neural networks. Biological Cybernetics, 46: 27-39.
Feldman, J. A., 1986, Neural Representation of Conceptual Knowledge. Technical Report 189, University of Rochester Computer Science.
Fisher, D. L., 1984, Central capacity limits in consistent mapping, visual search tasks: Four channels or more? Cognitive Psychology, 16: 449-484.
French, R. M., 1995, The Subtlety of Sameness: A Theory and Computer Model of Analogy-making (Cambridge MA: MIT Press).
French, R. M., and Thomas, E., 2000, Why localist connectionist models are inadequate for categorization. Behavioral and Brain Sciences, 23: 477.
Gentner, D., and Toupin, C., 1986, Systematicity and surface similarity in the development of analogy. Cognitive Science, 10: 227-300.
Gerstner, W., Kreiter, A. K., Markram, H., and Herz, A. V. M., 1997, Neural codes: firing rates and beyond. Proceedings of the National Academy of Sciences, USA, 94: 12740-12741.
Gevins, A., Smith, M. E., McEvoy, L., and Yu. D., 1997, High-resolution EEG mapping of cortical activation related to working memory: effects of task difficulty, type of processing, and practice. Cerebral Cortex, 7: 374-385.
Ghose, G. M., and Maunsell, J., 1999, Specialized representations in visual cortex: a role for binding. Neuron, 24: 79-85.
Glass, L., and Mackay, M. C., 1988, From Clocks to Chaos (Princeton, NJ: Princeton University Press).
Gray, C. M., König, A., Engel, A. K., and Singer, W., 1989, Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature, 338: 334-337.
Grossberg, S. and Grunewald, A., 1997, Cortical synchronization and perceptual framing. Journal of Cognitive Neuroscience, 9: 117-132.
Halford, G. S., Bain, J. D., and Maybery, M. T., 1984, Does a concurrent memory load interfere with reasoning? Current Psychological Research and Reviews, 3: 14-23.
Halford, G. S., and Wilson, W. H., 1980, A category theory approach to cognitive development. Cognitive Psychology, 12: 356-411.
Halford, G. S., Wilson, W. H., Guo, J., Gayler, R. W., Wiles, J., and Stewart, J. E. M., 1994, Connectionist implications for processing capacity limitations in analogies. In K. J. Holyoak and J. A. Barnden (eds) Advances in Connectionist and Neural Computation Theory, Volume 2: Analogical Connections (Norwood, NJ: Ablex), pp. 363-415.
Hebb, D. O., 1949, The Organization of Behavior (New York: Wiley).
Henderson, J. B., 1994, Description Based Parsing in a Connectionist Network. Doctoral dissertation, University of Pennsylvania.
Hinton, G. E., McClelland, J. L., and Rumelhart, D. E., 1986, Distributed representations. In D. E. Rumelhart and J. L. McClelland (eds) Parallel Distributed Processing Volume 1 (Cambridge, MA: MIT Press), pp. 77-109.
Hofstadter, D. R., 1984, The Copycat Project: An Experiment in Nondeterminism and Creative Analogies. AI Memo No. 755, Massachusetts Institute of Technology.
Horn, D., and Opher, I., 2000, Temporal segmentation and binding in oscillatory neural systems. In D. S. Levine, V. R. Brown and T. Shirey (eds) Oscillations in Neural Systems (Mahwah, NJ: Lawrence Erlbaum), pp. 201-216.
Hummel, J. E., and Biederman, I., 1992, Dynamic binding in a neural network for shape recognition. Psychological Review, 99: 480-517.
Hummel, J. E., and Holyoak, K. J., 1997, Distributed representation of structure: a theory of analogical access and mapping. Psychological Review, 104: 427-466.
Ishizuka, S., and Hayashi, H., 1996, Chaotic and phase-locked responses of the somatosensory cortex to a periodic medial lemniscus stimulation in the anesthetized rat. Brain Research, 723: 46-60.
Ivry, R. B., and Prinzmetal, W., 1991, Effect of feature similarity on illusory conjunctions. Perception Psychophysics, 49: 105-116.
Jensen, O., Idiart, M. A. P., and Lisman, J. E., 1996, Physiologically realistic formation of autoassociative memory in networks with theta/gamma oscillations: role of fast NMDA channels. Learning & Memory, 3: 243-256.
Jensen, O., and Lisman, J. E., 1996, Novel lists of 7 ± 2 known items can be reliably stored in an oscillatory short-term memory network: interaction with long term memory. Learning & Memory, 3: 257-263.
Jensen, O., and Lisman, J. E. 1998, An oscillatory short-term memory buffer model can account for data on the Sternberg task. The Journal of Neuroscience, 18: 10 688-10 699.
Kahana, M. J., Sekuler, R., Caplan, J. B., Kirschen, M., and Madsen, J. R., 1999, Human theta oscillations exhibit task dependence during virtual maze navigation. Nature, 399: 781-784.
Kanwisher, N. G., 1987, Repetition blindness: type recognition without token individuation. Cognition, 27: 117-143.
Konnerth, A., Tsien, R. Y., Mikoshiba, K., and Altman, J., 1996, Coincidence Detection in the Nervous System (Strasbourg: Human Frontier Science Programme).
Kroger, J. K., and Holyoak, K. J., 1997, What it means to be 'the same': the impact of relational complexity on processing efficiency. In Proceedings of the Nineteenth Annual Conference of the Cognitive Science Society (Mahwah, NJ: Lawrence Erlbaum), pp. 406-411.
Lange, T. E., 1992, Lexical and pragmatic disambiguation and re-interpretation in connectionist networks. International Journal of Man-Machine Studies, 36: 191-220.
Lange, T. E., and Dyer, M. G., 1989, High-level inferencing in a connectionist network. Connection Science, 1: 181-217.
Lisman, J. E., and Idiart, M. A. P., 1995, Storage of 7 ± 2 short-term memories in oscillatory subcycles. Science, 267: 1512-1515.
Löwel, S., and Singer, W., 1992, Selection of intrinsic horizontal connections in the visual cortex by correlated neuronal activity. Science, 255: 209-212.
Luck, S. J., and Vogel, E. K., 1998, Response from Luck and Vogel. Trends in Cognitive Sciences, 2: 78-80.
Maass, W., 1996, Networks of spiking neurons: The third generation of neural network models. Technical Report. Technische Universitaet Graz.
Maass, W., and Bishop, C. M., 1999, Pulsed Neural Networks (Cambridge, MA: MIT Press).
Maass, W., and Ruf, B., 1999, On computation with pulses. Information & Computation, 148: 202-218.
MacKay, W. A., 1997, Synchronized neuronal oscillations and their role in motor processes. Trends in Cognitive Sciences, 1: 176-183.
Mani, D. R., and Shastri, L., 1993, Reflexive reasoning with multiple instantiation in a connectionist reasoning system with a type hierarchy. Connection Science, 5: 205-242.
McCarthy, J., 1988, Epistemological challenges for connectionism. Behavioral and Brain Science, 11: 44.
McClelland, J. L., 1986, The programmable blackboard model of reading. In J. L. McClelland and D. E. Rumelhart (eds) Parallel Distributed Processing Volume 2 (Cambridge, MA: MIT Press), pp. 122-169.
McClelland, J. L., and Rumelhart, D. E., 1989, Exploration in Parallel Distributed Processing (Cambridge, MA: MIT Press).
Murthy, V. N., and Fetz, E. E., 1992, Coherent 25-to 35-Hz oscillations in the sensorimotor cortex of awake behaving monkeys. Proceedings of the National Academy of Sciences, USA, 89: 5670-5674.
Nakamura, K., Mikami, A., and Kubota, K., 1992, Oscillatory neuronal activity related to visual short-term memory in monkey temporal pole. Neuroreport, 3: 117-120.
Nelson, J. I., Salin, P. A., Munk, M. H. J., Arzi, M., and Bullier, J., 1992, Spatial and temporal coherence in cortico-cortical connections: a cross-correlation study in area 17 and 18 in the cat. Visual Neuroscience, 9: 21-38.
O'Keefe, J., and Burgess, N., 1999, Theta activity, virtual navigation and the human hippocampus. Trends in Cognitive Sciences, 3: 403-406.
Park, N. S., Robertson, D., and Stenning, K., 1995, An extension of the temporal synchrony approach to dynamic variable binding an a connectionist inference system. Knowledge Based Systems, 8: 345-358.
Phillips, W. A., and Singer, W., 1997, In search of common foundations for cortical computation. Behavioral and Brain Sciences, 20: 657-722.
Pomerleau, D. A., 1987, The Meta-generalized Delta Rule: A New Algorithm for Learning in Connectionist Networks. Technical Report CMU-CS-87-165. Carnegie-Mellon University.
Roelfsema, P. R, Engel, A. K., König, P., and Singer, W., 1996, The role of neuronal synchronization in response selection: a biologically plausible theory of structured representations in the visual cortex. Journal of Cognitive Neuroscience, 8: 603-625.
Samsonovich, A., and McNaughton, B. L., 1997, Path integration and cognitive mapping in a continuous attractor neural network model. The Journal of Neuroscience, 17: 5900-5920.
Shallice, T., and Warrington, E. K., 1970, Independant functioning of verbal memory stores: a neuropsychological study. Quarterly Journal of Experimental Psychology, 22: 261-273.
Shastri, L., and Ajjanagadde, V., 1993, From simple associations to systematic reasoning: a connectionist representation of rules, variables and dynamic bindings using temporal synchrony. Behavioral and Brain Science, 16: 417-494.
Shepherd, G. M., 1994, Neurobiology, 3rd edn (Oxford: Oxford University Press).
Singer, W., 1993, Synchronization of cortical activity and its putative role in information processing and learning. Annual Review of Physiology, 55: 349-374.
Singer, W., 1995, Synchronization of neuronal responses as a putative binding mechanism. In A. Arbib (ed.) The Handbook of Brain Theory and Neural Networks (Cambridge, MA: MIT Press), pp. 960-964.
Singer, W., Engel, A. K., Kreiter, A. K., Munk, M. H. J., Neuenschwander, S., and Roelfsema, P. R., 1997, Neuronal assemblies: necessity, signature and detectability, Trends in Cognitive Sciences, 1: 252-261.
Singer, W., and Gray, C. M., 1995, Visual feature integration and the temporal correlation hypothesis. Annual Review of Neuroscience, 18: 555-586.
Slawinska, U., and Kasicki, S., 1995, Theta-like rhythm in depth EEG activity of hypothalamic areas during spontaneous or electrically induced locomotion in the rat. Brain Research, 678: 117-126.
Smolensky, P., 1990, Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46: 159-216.
Sougné. J. P., 1996, A connectionist model of reflective reasoning using temporal properties of node firing. In Proceedings of the Eighteenth Annual Conference of the Cognitive Science (Mahwah, NJ: Lawrence Erlbaum), pp. 666-671.
Sougné, J. P., 1998a, Connectionism and the problem of multiple instantiation. Trends in Cognitive Sciences, 2: 183-189.
Sougné, J. P., 1998b, Period doubling as a means of representing multiply instantiated entities. In Proceedings of the Twentieth Annual Conference of the Cognitive Science Society (Mahwah, NJ: Lawrence Erlbaum), pp. 1007-1012.
Sougné, J. P., 1998c, Solving Problems Requiring More than 2 instantiations with Infernet. Université de Liège (http://www.fapse.ulg.ac.be/Lab/cogsci/jsougne/TR-98-2.pdf).
Sougné, J. P., 1999, INFERNET: A Neurocomputational Model of Binding and Inference. Doctoral dissertation, Université de Liège (http://www.fapse.ulg.ac.be/Lab/cogsci/jsougne/JSougneThesis. pdf).
Sougné, J.P., 2000, Simulating conditional reasoning containing negations: a computer model and human data. In L. R. Gleitman and A. K. Joshi, (eds) Proceedings of the Twenty-Second Annual Conference of the Cognitive Science Society (Mahwah, NJ: Lawrence Erbaum), pp. 918-923.
Sougné, J. P., 2001, A learning algorithm for synfire chains. In R. M. French and J. P. Sougné (eds) Proceedings of the Sixth Neural Computation and Psychology Workshop: Evolution, Learning, and Development (London: Springer).
Sougné, J., and French, R. M., 1997, A neurobiologically inspired model of working memory based on neuronal synchrony and rythmicity. In J. A. Bullinaria, D. W. Glasspool and G. Houghton (eds) Proceedings of the Fourth Neural Computation and Psychology Workshop: Connectionist Representations (London: Springer), pp. 155-167.
Sun, R., 1992, On variable binding in connectionist networks. Connection Science, 4: 93-124.
Tanaka, K., 1996, A continuous map of higher-level visual features of objects in monley inferotemporal cortex. In A. Konnerth, R. Y. Tsien, K. Mikoshiba and J. Altman (eds) Coincidence Detection in the Nervous System (Strasbourg: Human Frontier Science Programme), pp. 143-151.
Terrace, H. S., 1991, Chunking during serial learning by a pigeon: I. Basic evidence. Journal of Experimental Psychology: Animal Behavior Processes, 17: 81-93.
Touretsky, D. S., and Hinton, G. E., 1988, A distributed connectionist production system. Cognitive Science, 12: 423-466.
Treisman, A., and Gormican, S., 1988, Feature analysis in early vision: evidence from search asymmetries. Psychological Review, 95: 15-48.
von der Malsburg, C., 1981, The Correlation Theory of Brain Function. Internal Report 81-2, MPI für Biophysikalische Chemie, Göttingen. Reprinted in K. Schulten and H. J. van Hemmen (eds) (1994) Models of Neural Networks, 2nd edn (Berlin: Springer).
von der Malsburg, C., 1981, The Correlation Theory of Brain Function. Internal Report 81-2, MPI für Biophysikalische Chemie, Göttingen. Reprinted in K. Schulten and H. J. van Hemmen (eds) (1994) Models of Neural Networks, 2nd edn (Berlin: Springer).
Wang, X., and Rinzel, J., 1995, Oscillatory and bursting properties of neurons. In A. Arbib (ed.) The Handbook of Brain Theory and Neural Networks (Cambridge, MA: MIT Press), pp. 686-691.
Wilson, M., and Shepherd, G. M., 1995, Olfactory cortex. In A. Arbib (ed.) The Handbook of Brain Theory and Neural Networks (Cambridge, MA: MIT Press), pp. 669-673.
Winson, J., 1978, Loss of hippocampal theta rhythm results in spatial memory deficit in the rat. Science, 201: 160-163.
Yantis, S., 1992, Multi-element visual tracking: attention and perceptual organization. Cognitive Psychology, 24: 295-340.