Neural Modeling

After attending our local SeattleAI meeting and discussing neural network frameworks I decided to put together my notes on this subject.

What would one need to know to do neural modeling?

This paper (Introduction to Computational Neural Modeling for Computer Scientists and Mathematicians) provides a list of principles that govern well-practiced neural modeling:

• Knowledge of neuroscience principles
• Knowledge of computational time and space complexities
• Awareness of computational resources
• Well-researched neurological parameters
• Well-planned and implemented model design
• Documentation discipline (both parameter sources and coding)
• Knowledge of statistical analysis techniques
• Patience

Neuron

See Neural Modeling: Neuron

Synapse

• See Neural Modeling: Synaptic Plasticity
• See Neural Modeling: Synaptic Connectivity
• Post synaptic potential (PSP) models
• Neurotransmitters (only four most significant types out of hundreds discovered):
  • AMPA -- fast excitatory
  • NMDA -- slow excitatory
  • GABAA -- fast inhibitory
  • GABAB -- slow inhibitory
• Transmission delays
• Plasticity: presynaptic/postsynaptic
• Operations: AND, OR, XOR, NOT, SUM, MUL, MAX

Learning

• Learning algorithms:
  • Synaptic weight changes:
    • Hebb's rule: if S sends a pulse at time t and R fires at time t+1, then that synapse becomes more effective at firing R in the future
    • Milner's modification: if S sends a pulse at time t and R does not fire at time t+1, then that synapse becomes less effective in firing R in the future
  • Connection changes (more/less connections with a particular neuron)
  • Neuron changes (birth/death)
• Learning related problems:
  • Under-/over-learning; under-/over-fitting
  • Catastrophic interference
• Sensitization/habituation
• Types of learning: autonomous, continuous, on-line, supervised/unsupervised

Initial configuration

Architecture

• Input/output
  • Static/dynamic
  • Various types of sensory input and motor output
• Neural circuits
• Modular (Biologically Inspired Modular Neural Networks)

Simulation

• Time-driven vs. event-driven
• See Visualization

Overall models

• Rodney Cotterill: Co-operation of the basal ganglia, cerebellum sensory cerebrum and hippocampus: possible implications for cognition, consciousness, intelligence and creativity
• Peter F. C. Gilbert: An outline of brain function
• Yuri I. Arshavsky: Cellular and network properties in the functioning of the nervous system: from central pattern generators to cognition

Modeling tools and simulators

• See Tools and Simulators