Chris Chatman on his weblog portraits a gloomy picture awaiting biologically accurate neural modeling:

By most estimates, there are 100 billion neurons in the brain. Some neurons are known to have more than 1,000 dendrites, and up to about 1,000 different branchings of their axons. There are some 50 known neurotransmitters, and who knows how many other neuromodulators may exist (hormones, neural growth factors, neurosteroids). There are also many different receptor types for each of the neurotransmitters. A conservative estimate of the number of interactions you'd have to model to be biologically accurate is somewhere around 225,000,000,000,000,000 (225 million billion).

While this is definitely true, I'm almost equally split between two positions. On one side, the situation may be even more hopeless if you agree with Eugene Izhikevich's polychronization theory:

...spiking networks with delays have more groups than neurons. Thus, the system has potentially enormous memory capacity and will never run out of groups, which could explain how networks of mere 10¹¹ neurons (the size of the human neocortex) could have such a diversity of behavior. (p.270)

At the same time, I would not try to set a goal of building/modeling/simulating something that can exhibit general purpose intelligence; after all, you don't expect to have a conversation about weather with one of the participants of the Grand Challenge. I'm more interested in building brains that demonstrate special purpose intelligence. After all, there are "only" 20000 nerve cells in Aplysia, which is a much more manageable size to model.

I'm keenly interested in modeling the smallest and simplest brain that shows signs of intelligence (defined in Cotterill's terms).

In addition to all good things about the Simple Model of Spiking Neurons, there are few weaknesses that I observed while working on implementation of the model:

• The model appears to be sensitive to large currents (for some types of neurons); there is probably something wrong with my implementation of the model
• The model requires very small time step for some types of neurons (0.05ms for low threshold spiking (LTS) and fast spiking (FS) neurons); there is probably not much that can be done about this, but it definitely creates challenges with simulating those types of neurons as it requires to implement various time steps for different types of neurons to speed up the simulation. What's interesting, the previous version of the model had the same time step (0.5ms) for all types of neurons.

update 2005/12/21: According to the author of the model (private email):

There is indeed a problem with stability when the injected current, I, into a thalamic cell is too strong. You may interpret this as an artifact of the model, or as the sign that such a strong current can kill the cell.

update 2005/12/22: Again, from the author of the model on the same problem (private email):

What happens is that the peak of the spike and the voltage reset variables meet, periodic spiking disappears, and the voltage variable drops. This behavior resembles the "excitation block" behavior of many neurons (no spiking when the input current is too strong), though the mechanism may be different.

update 2005/12/22: And here is the recomendation on how to fix this:

...use the following formula for the after-spike reseting of u: u <- min(u+50,530). This way, u never goes above 530, and the the spike cutoff and afterspike reset values never intersect.

Here is the result and the modified script.

After finishing playing with excitatory neurons I turned to inhibitory neurons and after several unsuccessful attempts realized that I'll probably spend more time getting them right than I was planning on. Even though I tried many different sets of parameters (based on parameters documented in Izhikevich's book Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting) I just couldn't get it to work properly. I sent an email to the author, who kindly provided his MATLAB scripts that he used to model fast spiking (FS), late spiking (LS), and thalamic interneurons (TI) neurons. The reason for most of my problems was quite simple: I was using the wrong time step. For example, this is how the diagram looks for similation of one fast spiking (FS) neuron with 0.5ms time step:

and here is how it looks with the correct time step of 0.05ms:

Even though this fixed most of the problems there were still few I didn't know what to do about. For example, the model for thalamic interneurons was unstable with large currents (even with the current just slightly larger than showed in the book: 258pA vs. 255pA):

I also ended up modifying parameters for the low threshold spiking (LTS) neurons; I took c and d parameters from the model for fast spiking neurons (based on the older version of the model). Here is how the result looks like:

And here is the script that I used to generate all these diagrams.

As I wrote in my last post on neural modeling there are several things that I like about the model of spiking neurons proposed by Eugene Izhikevich (see also WHICH-MODEL-TO-USE for comprehensive comparison with other available models):

1. It's fast: only 13 FLOPS versus 5 FLOPS for Integrate-and-Fire and 1200 FLOPS for the Hodgkin-Huxley model
2. It's biologically plausible: it exhibits the same neuro-computational properties as the most complex, Hodgkin-Huxley model
3. It allows to model various types of neurons by changing four (in the latest revision seven) parameters
4. It has no fixed threshold or absolute refractory period: these are properties rather than parameters of the model and depend on the type of a neuron. Based on the history of the membrane potential prior to the spike, the threshold potential can be as low as -55 mV or as high as -40 mV.

Izhikevich proposes classification of all neuron types as resonator/integrator (based on presence of subthreshold oscillations) and as being bistable/monostable (based on co-existance of resting and spiking states), which, in combination, make four groups. By using this classification he is able to derive several neuro-computational properties of those types of neurons:

• Inhibition impedes spiking in integrators, but can promote it in resonators (this came as news to me; I was under the impression that inhibition always inhibit spiking)
• Integrators have all-or-none spikes while resonators may not
• Integrators have well-defined threshold while resonators may not
• Integrators integrate, resonators resonate. This means integrators prefer high-frequency inputs; the higher the frequency, the sooner they fire. By contrast, the response of the resonator neuron depends on the frequency content of the input

Most cortical pyramidal neurons (including regular spiking (RS), and chattering (CH) neurons shown in the previous post) are integrators. Most cortical inhibitory neurons are resonators. According to the author of the model, a good neuronal model must reproduce not only electrophysiology but also bifurcaiton dynamics of neurons.