After attending our local SeattleAI meeting and discussing neural network frameworks I decided to put together my notes on this subject.

What would one need to know to do neural modeling?

This paper (Introduction to Computational Neural Modeling for Computer Scientists and Mathematicians) provides a list of principles that govern well-practiced neural modeling:

• Knowledge of neuroscience principles
• Knowledge of computational time and space complexities
• Awareness of computational resources
• Well-researched neurological parameters
• Well-planned and implemented model design
• Documentation discipline (both parameter sources and coding)
• Knowledge of statistical analysis techniques
• Patience

Neuron

See Neural Modeling: Neuron

Synapse

• See Neural Modeling: Synaptic Plasticity
• See Neural Modeling: Synaptic Connectivity
• Post synaptic potential (PSP) models
• Neurotransmitters (only four most significant types out of hundreds discovered):
  • AMPA -- fast excitatory
  • NMDA -- slow excitatory
  • GABAA -- fast inhibitory
  • GABAB -- slow inhibitory
• Transmission delays
• Plasticity: presynaptic/postsynaptic
• Operations: AND, OR, XOR, NOT, SUM, MUL, MAX

Learning

• Learning algorithms:
  • Synaptic weight changes:
    • Hebb's rule: if S sends a pulse at time t and R fires at time t+1, then that synapse becomes more effective at firing R in the future
    • Milner's modification: if S sends a pulse at time t and R does not fire at time t+1, then that synapse becomes less effective in firing R in the future
  • Connection changes (more/less connections with a particular neuron)
  • Neuron changes (birth/death)
• Learning related problems:
  • Under-/over-learning; under-/over-fitting
  • Catastrophic interference
• Sensitization/habituation
• Types of learning: autonomous, continuous, on-line, supervised/unsupervised

Initial configuration

Architecture

• Input/output
  • Static/dynamic
  • Various types of sensory input and motor output
• Neural circuits
• Modular (Biologically Inspired Modular Neural Networks)

Simulation

• Time-driven vs. event-driven
• See Visualization

Overall models

• Rodney Cotterill: Co-operation of the basal ganglia, cerebellum sensory cerebrum and hippocampus: possible implications for cognition, consciousness, intelligence and creativity (or here) and MIND-A-MOVING-STORY
• Peter F. C. Gilbert: An outline of brain function
• Yuri I. Arshavsky: Cellular and network properties in the functioning of the nervous system: from central pattern generators to cognition

Modeling tools and simulators

• See Tools and Simulators

From Eugene Izhikevich's website:

On October 27, 2005 I finished simulation of a model that has the size of the human brain. The model has 100,000,000,000 neurons (hundred billion or 10¹¹) and almost 1,000,000,000,000,000 (one quadrillion or 10¹⁵) synapses. It represents 300×300 mm² of mammalian thalamo-cortical surface, specific, non-specific, and reticular thalamic nuclei, and spiking neurons with firing properties corresponding to those recorded in the mammalian brain. The model exhibited alpha and gamma rhythms, moving clusters of neurons in up- and down-states, and other interesting phenomena.
One second of simulation took 50 days on a beowulf cluster of 27 processors (3GHz each). Why did I do that?

This sounds like a great achievement, but I've always been interested in modeling a smallest, simplest, and slowest brain that exhibits intelligence. How many neurons and connections would it have? What modules and components are essential and what are not? How much of intelligence would be lost because of its size and simplicity? How fast can it be trained to do something useful? What properties would need to be wired in and what properties would emerge?