Biological Neural Networks

Question 1

What is the goal of neural science?

Answer 1

To understand how the flow of electrical signals through neural circuits gives rise to mind.

Question 2

What is computational neuroscience?

Answer 2

Computational neuroscience is the field of study in which mathematical tools are used to investigate brain function.

• can incorporate electrical engineering, computer science, physics, etc.

Question 3

What are the components of a (biological) neuron?

Answer 3

• soma: cell body containing the nucleus
• dendrites: nerve cell extensions that receive electrical impulses from other neurons through synapses
• axon: long and thin part which send electrical impulses to other neurons. Axons end at synapses. Axons are insulated by myelin.

Question 4

How are neurons functionally classified?

Answer 4

Neurons are functionally classified as sensory neurons, motor neurons, or interneurons (connecting two brain regions).

Question 5

What does the polarity of a neuron denote?

Answer 5

The polarity denotes the number of different processes that originate from the soma.

A multipolar neuron possesses a single axon and many dendrites, allowing for the integration of a great deal of information from other neurons.

Question 6

How does the basic electrophysiology of a neuron work?

Answer 6

Information processing and exchange is based on electrical signals carried by ions (Na⁺, K⁺, Cl⁻).

Question 7

What is the function of a neuron’s cell membrane?

Answer 7

The cell membrane of a neuron is a bilipid layer that insulates the inside of the cell from the outside, acting as a capacitor.

Question 8

What are ion channels?

Answer 8

Ion channels are proteins in the membrane that can open and close to allow for a passive exchange of ions.

• Selectivity: selective ion channels only let a specific type of ions pass
• Regulation: opening and closing of an ion channel can be gated by external forces

Question 9

What is the Resting Membrane Potential?

Answer 9

The resting voltage of a neuron across its membrane.

sum of the different ion flows between the inside and the outside of a neuron

Question 10

What is a (biological) neuron?

Answer 10

A neuron is an electrically excitable cell that communicates with other cells via specialized connections called synapses.

Question 11

How can the Resting Membrane Potential be calculated?

Answer 11

Approximation by computing the Nernst potential of potassium

Exact potential by taking into account all ion types (-> Goldman equation)

Question 12

What is an Action Potential (spike)?

Answer 12

An action potential occurs when the membrane potential of a neuron rapidly rises and falls (depolarizes): this depolarization then causes adjacent locations to similarly depolarize.

After some delay, the membrane voltage transiently falls back below the resting state (hyperpolarization)

ca. 2-4 ms

Question 13

What is the refractory period?

Answer 13

Directly after an action potential, the neuron must recover the resting membrane potential.

Two phases:

• absolute refractory period: dead time during which the neuron cannot emit any new spike
• relative refractory period: eliciting an action potential requires stronger stimuli

ca. 1-2 ms

Question 14

What are the different modes propagating action potentials along nerve cells?

Answer 14

• electrotonic conductance: passive conduction of charges along neurons with high speed. Strength decays with distance
• active regeneration of action potentials: charges cause repeated depolarization in the axon and regenerate the action potential. Slow propagation speed

Question 15

What is Myelin?

Answer 15

Myelin is a lipid-rich substance that form a sheath around neuron axons to insulate them and increase the rate at which action potentials are passed along the axon.

Question 16

What are the Nodes of Ranvier?

Answer 16

Gaps in the myelin sheath, where action potentials are regenerated.

Question 17

What is saltatory conduction?

Answer 17

Saltatory conduction (lat. saltare, to hop or leap) is the propagation of action potentials along myelinated axons from one node of Ranvier to the next node, increasing the conduction velocity of action potentials.

Question 18

How are action potentials “digital” events that differ considerably from most other processes in biology?

Answer 18

• threshold: membrane voltage crosses ~-50 mV
• all-or-none event: shape of the spike does not depend on history
• propagation, self-regeneration
• refractory period

Question 19

What is a synapse?

Answer 19

A synapse is a structure that permits a neuron to pass an electrical or chemical signal to a connected neuron.

Every synapse consists of both a presynaptic and a postsynaptic structure.

Question 20

What are the two types of synapses?

Answer 20

• electrical synapses: presynaptic and postsynaptic cell membranes are connected by special channels called gap junctions that are capable of passing an electric current. Main advantage: rapid transfer of signals
• chemical synapses: release of neurotransmitter by the presynaptic cell binds to receptors located in the membrane of the postsynaptic cell.
• • chemical synapses can be excitatory or inhibitory, based on the neurotransmitter
• • synapses can change their strength (learning)

Question 21

What are glial cells?

Answer 21

Glia / Glial cells are non-neuronal cells that do not produce electrical impulses.

There are about ten times more glia than neurons.

Two main types:

• Macroglia (most common): formation of myelin sheath, nutrition of neurons
• Microglia: immune system cells

Question 22

What are the two main classes of glia?

Answer 22

• Macroglia (most common): formation of myelin sheath, nutrition of neurons
• Microglia: immune system cells

Question 23

What are different classes of biological neuron models?

Answer 23

• detailed compartmental models: detailed neuron morphology (geometry) based on anatomical reconstructions. Continuous geometrical structure discretized into compartments
• reduced compartmental models: only a few dendritic compartments
• single-compartment models (point neuron models): ignore neuron morphology, only capture flow of ion currents and spike generation
• > most prominent class of neuron models, what we use
• cascade models: purely functional models that only represent abstract computations but no biological detail
• black box models: no computations, neurons as I/O system with defined statistic behavior

Question 24

What is the McCulloch-Pitts Model?

Answer 24

One of the first (1943) computational neuron models, based on digital threshold logic:

• a neuron can have both excitatory synapses eₖ and inhibitory synapses iₗ
• if the neuron receives input from any inhibitory synapse, the output is 0
• if no inhibitory synapse is active and the sum of active excitatory synapses crosses a threshold, the output is 1

The model is today considered the starting point of modern AI.

Question 25

What are General Analog Neuron Models?

Answer 25

Generalization of the simple McCulloch-Pitts model with synaptic weights and an activation function.

A(wᵀx + b)

-> Analog neuron models are generalized linear models!

Question 26

What are common activation functions?

Answer 26

Binary step / Heaviside, Logistic, tanh, ReLU, Exponential Linear Unit (ELU), Gaussian

Question 27

What are spiking neuron models?

Answer 27

Neuron models that incorporate time.

They model the electrophysiological properties of biological neurons using functionally equivalent electrical circuits:

• cell membrane = capacitor
• ion flow = electrical current
• opening and closing of ion channels = change of conductance (inverse of resistance)

➔ Biological neurons can be emulated using analog electrical circuits

Question 28

What is the Hodgkin-Huxley Model?

Answer 28

The Hodgkin–Huxley model (conductance-based model) is a set of nonlinear differential equations that approximates the electrical characteristics of neurons. It is a continuous-time dynamical system that can be described by a electrical circuit.

Question 29

What is the major drawback of the Hodgkin-Huxley Model?

Answer 29

Its computational complexity, mainly caused by the computation of the ionic currents.

Model reductions can be derived, based on two-dimensional differential equations (instead of four dimensions), e.g. the Morris-Lecar model or the FitzHugh-Nagumo model.

However, even reduced conductance-based neuron models are more detailed than required. They can be replaced by phenomenological models that capture the overall behavior of biological neurons without explicitly emulating the underlying mechanics.

Question 30

What are phenomenological models?

Answer 30

A phenomenological model is a scientific model that describes the empirical relationship of phenomena to each other, in a way which is consistent with fundamental theory, but is not directly derived from theory.

A phenomenological model forgoes any attempt to explain why the variables interact the way they do, and simply attempts to describe the relationship, with the assumption that the relationship extends past the measured values.

Question 31

What is the Leaky Integrate-and-Fire (LIF) Model?

Answer 31

The Leaky Integrate-and-Fire Model is a phenomenological neuron model which is based on a simplified electrical circuit with a capacitor and a resistor.

Input current I(t) = I_Resistor(t) + I_Capacitor(t)
= u(t)/R + C * du/dt

Dynamics function:
τₘ du/dt = −u(t) + R*I(t)

Question 32

What is the AdEx Model?

Answer 32

The adaptive exponential integrate-and-fire (AdEx) model is an extended phenomenological spiking neuron model that combines the capability of the Hodgkin-Huxley model (biologically realistic firing patterns) with the low computational complexity of other phenomenological models.

It is based on a two-dimensional differential equation.

Question 33

What is a spike train?

Answer 33

A spike train is a sequence of recorded times at which a neuron fires an action potential.

Question 34

What are Neural Codes?

Answer 34

Neural codes are encoding schemes to store data in spike trains.

• Rate Coding
• Time-To-First Spike Coding
• Coding by Synchrony or Correlation

Question 35

What is Rate Coding?

Answer 35

A neural code in which information is carried in the average spike rate within a certain time window (e.g. muscle excitation)

Question 36

What is Time-to-First Spike Coding?

Answer 36

A neural code in which information is encoded in the time of emission of the first spike after a reference point

e.g. time from stimulus to spike

Question 37

What is Coding by Synchrony or Correlation?

Answer 37

Coding by Synchrony or Correlation is a neural code based on activity patterns of a group of neurons (e.g. averaged overall activity)

It is based on the direct or indirect temporal coherence between spikes of individual neurons in a population.

Question 38

What are population models?

Answer 38

Population models are neural codes which only model the mean activity of a population, since in large populations, the contribution of a single neuron becomes more and more negligible.

Populations are modeled based on density functions that compute the probability that an arbitrary neuron has a certain internal state.

Question 39

How do spiking neurons compare to analog neurons?

• Model
• Communication
• Encoding of information
• Computational complexity
• Biological Plausibility

[IMPORTANT]

Answer 39

Spiking Neurons / Analog Neurons

• Model: dynamical system / activation function
• Communication: “digital” spikes / “analog” values
• Encoding of information: spike timing, rate, synchrony / real numbers
• Computational complexity: high / low
• Biological Plausibility: usually high / usually low

Question 40

What model is today considered the starting point of modern AI?

Answer 40

The McCulloch-Pitts Model

Question 41

What are the two approaches to learning?

Answer 41

• Experiment: how does nature do it? Perform experiment as a targeted question to nature, observe changes in the synaptic strength
• Theory: formulate a theory to describe neural networks, find a way to find the right weights

Question 42

What is Hebb’s rule?

Answer 42

“The general idea is that any two cells that are repeatedly active at the same time will tend to become ‘associated’ so that activity in one facilitates activity in the other.”

“what fires together, wires together”

Question 43

How is Hebbian learning modeled mathematically?

Answer 43

Changes in weight depend on the activity of the pre- and postsynaptic neuron and the weight itself:

dw/dt = F(w, vᵢ, vⱼ)

Taylor approximation:

dw/dt = Σₙ,ₘ cₙ,ₘ(w) · vᵢⁿ · vⱼᵐ

Note: no feedback required!

Question 44

What are long-term potentiation (LTP) and long-term depression (LTD)?

Answer 44

Long-term potentiation (LTP) is a persistent strengthening of synapses based on recent patterns of activity.

LTD is the opposite.

Question 45

What is STDP?

Answer 45

Spike-timing-dependent plasticity (STDP) adjusts the connection strengths between neurons based on the relative timing of the spikes.

The STDP process partially explains the activity-dependent development of nervous systems, especially with regard to long-term potentiation and long-term depression.

Question 46

What are the three generations of neuron models?

Answer 46

1. binary
2. real numbers
3. biologically realistic spikes

Question 47

What is the downside of perturbation learning?

Answer 47

It is very slow.

Question 48

How does supervised STDP work?

Answer 48

Simulate spike as usual, marking all weights that would have to be updated (eligibility traces). Check if output matches the label. Only if it matches, execute the STDP update!

Question 49

What is SpikeProp?

Answer 49

Backpropagation in SNN

Approaches:

• treating discontinuities as noise
• differentiable approximation

→ Better learning, worse performing neurons

Solution: train ANN, convert SNN (replace activation function with a spiking neuron model)

Question 50

What is the Upper Limit Problem?

Answer 50

Compare ReLU and Rate Coding. Problem: spiking neurons have a maximum frequency, ReLU has no upper limit

Solutions:

• normalization: calculate max value of ReLU neurons, normalize weights by that value
• constrain-than-train: limit network before training, e.g. clamped ReLU (1 if > 1)

Question 51

SNNs integrate signals over time. What is the advantage?

Answer 51

SNNs learn temporal features.

Some signals contain highly time correlated features (e.g. waves)

• > finding time pattern entails a much lower complexity
• > where ANNs lead to complex input space and big network space, SNNs can increase efficiency and performance

Question 52

What is a memristor?

Answer 52

A theoretical fourth elemental two-terminal circuit (Chua, 1971).

• resistance based on the previous amount of charge

Question 53

What are central pattern generators (CPG)?

Answer 53

Central pattern generators (CPGs) are biological neural circuits that produce rhythmic outputs in the absence of rhythmic input.

CPGs offload low-level control for rhythmic tasks such as walking or chewing from the brain; rhythmic motions are executed “automatically”.

Question 54

What are the four phases of an action potential?

Answer 54

1. resting state
2. depolarization
3. repolarization
4. refractory period

Question 55

What are two motor control functions performed by the spinal cord?

Answer 55

• spinal reflexes

- central pattern generators