Hopfield and attractor networks

Question 1

hopfield network short

Answer 1

a fully connected recurrent networke

Question 2

recurrent

Answer 2

each neuron is connected to every other neuron

Question 3

hopfield mechanism

Answer 3

stores memories (patterns of -1 and +1) by creating a kind of energy landscape. connections between neurons are set up in a symmetric way, meaning the influence from A to B is the same from B to A, which allows the network to have a well defined energy function, which always decreases as the network updates.

Instead of looking up a memory like a dictionary, the hopfield network dynamicaaly finds it based on similarity.op

Question 4

hopfield valley

Answer 4

its like an energy landscape, a hilly terrain with valleys. each valley represents a memory the network has learned. these valleys are the stable states of the network, once you flal into one, you stay there.

imagine you drop a ball somewhere in the landscape, not exactly in a valley but nearby (a noisy or incoplete version of a memory). the ball will roll downhill following the slope, unti it settles into the nearest valley.

Question 5

how is a hopfield model a dynamic system

Answer 5

• the state of each neuron can change at each step
• the network updates neuron values one at a time, or all together
• with every update, the total energy of the system decreases
• eventually the system stops changing - it has reached a stable point

so the dynamics of the system are the step by step changes in neuron activity, driven by rules that guarantee the system eventyally stops moving

Question 6

what is a dynamic system

Answer 6

something that evolves over time based on a set of rules

Question 7

attractor

Answer 7

neural networks that evolve their activity/dynamics over time to settle int ostable states called attractors

Question 8

discrete attractor

Answer 8

a single points or several discrete points form the stable states. can be visualizes as valleys in the energy landscpaesco

Question 9

continuous attractors (ring attractors)

Answer 9

a continuum of stable states. the possible attractor states are no longer discrete, but can vary continuously

Question 10

activity bump and how does it move

Answer 10

nearby cells excite eachother.

moving:
activity copy: the current position of the activity bymp is copied ito a hidden layer of neurons
movement signals: the hidden layer also receives movement signals
asymmetric projections: the hidden layer sends asymmetric projections back to the attractor layer, nudging the bump in the appropriate direction
conjunctive cells: the neurons in the hidden layer are conjunctive cells that combine the current bump position and the movement signal, encoding both together
steering the bump: this combinatin of position and movement allows the netwrok to update the head direciton smoothly, following the intendend movement

Question 11

spurious attractors

Answer 11

with too many memories, the network might recall mictures of patterns rather than a single correct one

Question 12

max number of memories

Answer 12

N_patterns = 0.138 times N_neurons

Question 13

continous ring attractor characteristics

Answer 13

1. layer of conjunctive cells (head direction x angluar velocity
2. pairwise correlations between head direction neurons should be preserved across conditions
3. persistent activity: activity should not die off when externl input is removed
4. population activity should be constrained to a ring-like manifold (ring attractor)
5. specific connectivity between the head direction cells (local excitation, global inhibition)