Symbol Polychronization

So far I've been focusing on the modeling of individual neurons and their networks, but after reading more on thought processes in Godel, Escher, Bach (pp.348-50), I thought it might be interesting to look at the level of neural complexes that represent symbols in the brain, rather than at the level of individual neurons and signals between them.

There are many questions that come to mind concerning symbols and their representations in the brain:

  1. How many neurons need to be connected to form a neural complex?
  2. Can a neuron belong to more than one complex?
  3. How many complexes can a single neuron belong to?
  4. How many neurons need to be activated for the symbol to become active?
  5. Are there certain core neurons for each symbol that become active every time the symbol is activated?
  6. How many neurons can two symbols share?
  7. How similar are complexes that represent the same symbol in different brains?
  8. What regions of the brain these complexes extend into?

Then Hofstadter offers a hypothesis about symbol representation in the brain (pp.356-7):

...overlapping and completely tangled symbols are probably the rule, so each neuron, far from being a member of a unique symbol, is probably a functioning part of hundreds of symbols.

...in order to distinguish one symbol's activation from that of another symbol, a process must be carried out which involves not only locating the neurons that are firing, but also identifying very precise details of the timing of the firing of those neurons. Thus perhaps several symbols can coexist in the same set of neurons by having different characteristic neural firing patterns.

These ideas come very close to ideas expressed by Izhikevich in his paper on polychronization, which is defined as reproducible time-locked but not synchronous firing patterns with millisecond precision. Neurons are spontaneously organized into polychronous groups by the spike-timing-dependent plasticity (STDP) changes that select conduction delays to allow the groups to form. As each neuron participates in many groups, firing with one group at one time and with another group at another time, the number of coexisting polychronous groups could be far greater than the number of neurons in the network and even greater than the number of synapses.

What is the significance of polychronous groups? We hypothesize that polychronous groups could represent memories and experience. (p.261)

...the system has potentially enormous memory capacity and will never run out of groups, which could explain how networks of mere 1011 neurons (the size of the human neocortex) could have such a diversity of behavior. (p.270)

Let's entertain the idea that polychronous groups are, in fact, neural complexes described above that represent symbols in the brain. Now we can try to answer questions on the list above:

  1. How many neurons need to be connected to form a neural complex? Probably as few as five. Izhikevich's paper shows five neurons with optimized delays that generate 14 groups.
  2. Can a neuron belong to more than one complex? Yes.
  3. To how many complexes can a single neuron belong? Potentially more than the number of synapses it has.
  4. How many neurons need to be activated for the symbol to become active? From the paper (p.267): "When a group is activated, whether in response to a particular stimulation or spontaneously, it rarely activates entirely. Typically, neurons at the beginning of the group polychronize, that is, fire with the precise spike-timing pattern imposed by the group connectivity, but the precision fades away as activation propagates along the group. As a result, the connectivity in the tail of the group does not stabilize, so the group as a whole changes."
  5. Are there certain core neurons for each symbol that become active every time the symbol is activated? According to this model, there are always some neurons that need to be activated for a group to become active.
  6. How many neurons can two symbols share? I don't think there are any limits. Two groups can share all their neurons and have separate activation patterns.
  7. How similar are complexes that represent the same symbol in different brains? Not similar at all.
  8. What regions of the brain these complexes extend into? I can't give you any definitive answer, but probably not all. Can it be that we have two types of regions in the brain: some that are formed with specific neuronal circuits and others that have randomly connected networks of neurons that self-organize into groups based on their inputs?

After thinking about some of the answers I came up with more questions:

  1. Can groups be replicated? It seems like all we need to do is to provide the same input and then the group(s) emerge after STDP selects proper delays. It may not be the same group, but it will represent the same concept in a different place in the brain. Or will it? Is a symbol defined by its neuronal representation or by its connections with other symbols?
  2. How do we think of something we never saw before? How do we connect those symbols to form a new concept? Do they need to form one group? Do they need to be active at the same time? How do we imagine three dogs in a teacup? Do we imagine three dogs somewhere else and then make it look like a teacup when we focus our attention on it? More on this from Godel, Escher, Bach (p.362): "As we imagine a hypothetical event, we bring certain symbols into active states -- and depending on how well they interact (which is presumably reflected in our comfort in countinuing the train of thought), we say the event 'could' or 'could not' happen."
  3. How do we keep symbols active? How do we morph them into something they are not? Do we create a new symbol based on the existing one every time we do this or do we temporarily change the existing symbol?

It's also interesting to note that, while any particular neuron can be a part of many groups, it can't participate in all those groups at the same time; at any given moment it can only be a member of at most one (?) group.

1 Comment

The Izhikevic model is a large system of difference-differential equations. The phase space is extremely large. Moreover, it has a complicated structure imposed by the delays. As a subset of R^n, it is already beyond comprehension. The theory of dynamical systems does not give much insight on such systems.

My own attempts at building an OCR System ( how old-fashioned ) that replicates the phenomenon of hyperacuity have no resemblance to "computing with spikes". On a more abstract level, however, there might be a connection: An activated polychronous group may be a section in a suitable sheaf.

The questions Q1,Q2,Q3 above all have answers in the context of my pattern engine. It is some kind of hacker approach that has been overlooked or underestimated for a very long time. But then, the pioneers could not even dare to think about throwing some twenty megabytes of fast digital memory on the problem of cognition.

Leave a comment

what will you say?
(required)
(required)

About

I am Paul Kulchenko.
I live in Kirkland, WA with my wife and three kids.
I do consulting as a software developer.
I study robotics and artificial intelligence.
I write books and open-source software.
I teach introductory computer science.
I develop a slick Lua IDE and debugger.

Recommended

Close