Exam V2 Flashcards

Question

What is a central pattern generator and **give an example of it?** = first paragraph (9)

Answer 1

The heartbeat control system of medicinal leech has been studies for three decades as CPG. The leech has two tubular hearts running along the length of the body and moves blood through a closed circulatory system. The beating pattern of leeches (beat period of 4s to 10s) is asymmetric with one heart generating high systolic pressure through front-directed peristaltic wave (peristaltic coordination mode) along its length and another heart generating low systolic pressure through near-synchronous constriction (synchronous coordination mode) along its length. The peristaltic heart moves blood forward. As compared to the peristaltic heart, the synchronous heart has been hypothesised to push blood into peripheral blood circulation and supports rearward blood flow. After about 20 to 40 heart beats (switch period ~ 100 – 400 s) the heart switches roles. The two heart tubes leech has receives excitatory input from ipsilateral member of a pair of segmental heart motor neurons (HE) which is located in each midbody segmental ganglion. The firing pattern of HE neurons (i.e., fictive motor pattern) is bilaterally asymmetric with motor neurons on one firing rear-to-front progression while those on other side fire nearly synchronously with appropriate side-to-side coordination of these two firing pattern (i.e., firing pattern switch). The HE neurons are controlled and coordinated by heartbeat CPG through rhythmic inhibitory drive.

Answer 2

There are nine pairs of identified segmental heart interneurons (HN) (plus one identified pair) that compose of CPG. The core CPG consists of 7 pairs of interneurons located in first seven midbody ganglia of nerve cord and indexed by ganglion number and body side (HN(L,1) – HN(R,7)). The rhythmic activity in CPG network is paced by highly interconnected timing consisting of coordination (HN(1) and HN(2) interneurons) and osciliatory interneurons (HN(3) and HN(4) interneurons). The firing pattern of interneurons of core CPG is also bilaterally asymmetric like HE neurons with appropriate side to side coordination. The asymmetry of firing pattern is not permanent as there are regular side to side switches in CPG network as peristaltic and synchronous pattern in HN underlie changes in both motor pattern and rhythmic constriction pattern in heart tubes. The switches in coordination is mediated by HN(5) switch interneuron which link the timing of network to middle premotor neurons by bilateral inhibitory connections; only one of the pair of interneurons rhythmically active at a time and other is silent. The premotor interneurons and motor neurons on one side of the active switch interneurons are coordinated synchronously while those on other side of silent switch interneurons are coordinated peristaltically.

Answer 3

Research has shown that in a task where letters are presented visually, participants show errors that indicate that information is acoustically coded. For example, participants replace T for G (sound similar) instead of Q for G (appearance of letters look similar). Similarly, participants found that recalling a wordlist more difficult for similar sounding words and not semantically related words such as recalling ‘rice’ instead of ‘ice’ and not recalling ‘frost’. Further research also shown that repeating nonsense syllables disrupts the phonological memory. All these research discussed above indicates that the working memory (WM) system is not unitary but a multi-component system with modality specific components, each can be damaged separately.

Answer 4

Research has shown that WM components can be damaged separately. As research shown that damage to Brodmann areas 44 and 40 means individuals can not hold strings or words in their memory or mind and have deficit in the rehearsal process of phonological loop. Research also shown participants have visuospatial sketchpad WM deficits as damage/lesions to the parieto-occipital causes deficits in visuo-spatial WM for instance that participants with that damage have difficulties memorising and repeating a sequence of blocks the experimenter has touched.

Answer 5

The Lisman-Idiart model proposes that there is a neural mechanism called ADP that helps out in working memory (WM) maintenance. ADP stands for afterdepolarisation. Depolarisation occurs when the membrane potential increases and more likely to emit a new spike. The ADP is a positive ‘hump’ in membrane potential that is produced after a spike is emitted in the model.

Answer 6

In Lisman Idiart model they have network in which activity of prefrontal neurons is calculated by their equation of rate of change of membrane potential over time. In this equation, they have terms such as: 1) VOSC which is a sin function that causes fluctuation in membrane potential in background and, 2) Vinh which is a term added every time an action potential is fired by a presynaptic neuron. VOSC provides excitatory oscillatory input to the neurons. Vinh has a feedback inhibition circuit. If VOSC neuron fires a spike it excites all prefrontal neurons and eventually transmitted to Vinh neuron which inhibits all neurons including itself. The model assumes that the firing of each neuron in network represents one item in WM.

Answer 7

How ADP and neuronal network of Lisman and Idiart model work together to implement active rehearsal for content in WM is explained below. There is background oscillations in the Lisman and Idart model’s network. If we present a letter G for someone to remember then the neurons in the network can quickly create synaptic connections with neuron in phonological loop which encodes and represents the letter ‘G’. The part of the phonological loop that represents ‘G’ will make a particular neuron in network fire an action potential. This neuron will then inhibits itself and all the other neurons in the network via the feedback inhibition circuit. Then once next peak of osciliation comes around, ADP raises the membrane potential high enough for that neuron that represents letter ‘G’ to fire again. The ADP and oscillatory inputs maintain the spiking of that neuron.

Answer 8

The Lisman-Idiart model does not use a Hodgkin Huxuley model since they want to see how a neural mechanism of ADP (Afterdepolarisation) helps to carry out working memory (WM) maintenance. They do not need to think about which ion channel is in charge of ADP as they want to model ADP’s effect on membrane potential over time. The model does not give an explanation of ADP and use ADP to explain higher-level phenomenon. The aim of Lisman and Idiart’s model is to demonstrate what ADP can be used for and its effect on membrane potential. Therefore, a Hodgkin Huxley model is not needed as it will not fulfil the Lisman-Idiart’s aim of adding how ADP functions in terms of a network of neurons.

Answer 9

The model demonstrates how neuronal prosperities (ADP) and network structure (feedback inhibition and oscillatory input) work together to implement function A criticism is that the authors chosen parameters (e.g., oscillation frequency) to make the number 8 for capacity.

Answer 10

Head direction cells are predominantly found in large network of brain areas in Papez circuit (Taube, 2007) such as the: 1) entorhinal cortex, 2) the thalamus (lateral dorsal and anterior dorsal nuclei) and, 3) anterior dorsal thalamus. Head direction cells are also found in non-Papez circuit brain areas such as: 1) lateral dorsal thalamus, 2) dorsal striatum and, 3) medial precentral cortex.

Answer 11

The two types of cells that are important for spatial cognition is: 1) head direction (HD) cells and, 2) place cells. Receptive fields are areas at which simulation leads to a response of a specific sensory neuron. Different place cells and HD cells are distinguished by their different receptive fields. Place fields have a receptive field for spatial location which means a particular place neuron will fire most vigorously at a particular location in the environment. HD cells have a receptive field for head orientation which means they will have a specific head orientation at which a specific HD fires maximally.

Answer 12

The three uses of head-direction cells is that used for orientation which is very important for navigation. It is also used for grasping and pointing so if you want to reorient yourself and do some action like pointing somewhere in a specific direction. Finally, head-direction cells is used to define a point of view (human spatial cognition).

Answer 13

From manipulations of single-cell recordings, experiments found three defining properties of head-direction (HD) cells which is: 1) HD cells depend on vestibular input, 2) cue cards control angular turning and, 3) HD drift in darkness meaning that without any visual input the animal loses its sense of orientation. Stackman and Taube found HD cells depend on vestibular input (i.e., changes in direction, movement and position of head) as they found that neurotoxic lesions of vestibular labyrinth abolished HD cell signal up to three months post lesions. Taube also demonstrated cue cards control angular turning by recording HD cells in a cylinder which contains a prominent visual cue (e.g., white cue card) attached to the box. They rotated this important visual landmark which leads to a corresponding shift in preferred firing direction of HD cells. Thus, HD cells controlled by landmarks. Mizumori and Williams found HD cells drift in darkness as when rats are either blindfolded or placed in complete darkness then preferred direction of HD cells become less stable (disrupted) and begins to drift. The hypotheses from these 3 main defining properties of HD cells is that HD cells used for navigation and when the animal lost it way, HD cells have lost their stable directional tuning which makes them drift.

Answer 14

We can correct the drift we see in the head-direction (HD) cells when animal lost its way in dark by receiving feedback from visual cue. Visual cells are somewhere in visual cortex that provide feedback (i.e., providing synaptic inputs at particular orientations to specific HD cells). When animal sees a cue card ahead in a box, a specific visual cell will be active and give strong synaptic input to the appropriate and correct HD cells which allows the animal to orient itself.

Answer 15

As animal moves around in a box, our chosen neuron fires at varying rates depending on the heading. We sum all the activites of the neuron and divide by total time animal spend in box to get the tuning curve. The tuning curve of a single HD neuron has firing rate of a single HD neuron as a function of heading and data accumulated over time.

Answer 16

Head-direction (HD) cells preferred firing direction still fire maximally if the head is still at a certain head orientation and its firing is maintained briefly in darkness where individual receives no sensory information. This is done via the short-range excitatory synaptic connections and long-range inhibitory connections the ring of HDs have. The HD cell that fires and most active at a certain direction sustain their activity , even in darkness, by exciting itself as well as exciting neighbouring HD cells near them due to the short-range excitatory synaptic connections (recurrent connections) as well as having long-range inhibitory synaptic connections to distant HD cells to suppress their activity. There is close-range excitation and long-range inhibition for each HD neuron in the ring. Thus, symmetric short-range and long-range inhibition gives sustained activity of HD cell.

Answer 17

To turn your head to another direction, the activity pattern of ring of HD cells will need to be shifted along the line of neurons. These line of HD neurons active will have offset inhibition in the direction opposite of a turn and offset excitation in direction of a turn. These connections will be active only when the head is turning (dependent on velocity). These connections are doubled, one for clockwise and one for counter clockwise. To turn clockwise, nearby HD cells to the right will be excited along the line of neurons. To turn anti-clockwise, nearby HD cells to the left will be excited in line of neurons in HD ring. Thus, velocity dependent asymmetric excitation and inhibition gives capability to turn head and shift pattern of activity across ring of HD neurons.

Answer 18

The two possible behaviours when giving external stimulus to HD ring in Zhang 1996 HD ring continuous attractor network (CAN) is: 1) shift and, 2) reset. Zhang found that the internal direction maintained by HD cell networks is calibrated by external inputs from a local-view detector . If the activity of HD cell network is maintained at 180 degrees but heading is actually at 200 degrees then external input from the local-view detector induce a shift in its activity towards 200 degrees. Reset is when the excitation of HD cells in network is too far away from the actual orientation of heading then external input from local-view detector produce new estimate  this resets the heading.

Answer 19

The HD ring network is an example of a continuous attractor network because you can place the activity anywhere you wanted in line of HD neurons. The activity can be shifted and come to rest at a new position. The HD ring network can sustain its activity as connectivity pattern is same for each neuron when heading is still at a certain orientation.

Answer 20

All activity of HDCs converge to different locations and at the end only represent a subset of all possible orientations. The continous attractor becomes a discrete attractor. Certain HDC attract the activity bump. Discrete basins of attractions (circle)

Answer 21

- Continuous attractor is symmetric connections maintain the activity packet in place we can place the ball anywhere - Discrete attractor: Given a bit of time, the ball settles in one of several valleys (basins). - Locations between valley are unstable.  Is this what happens to HD during aging due to neuron loss?

Answer 22

The advantages of the HD ring CAN is that it is the best model of HD we have. It allows us to explain how internal sense of direction is coded and maintained. It does not make use of spikes, let alone ion channels (assuming all the information about head direction is encoded in firing rate of the network). It is also good case of rate-coded neurons

Answer 23

The disadvantage of the HD CAN is that how the brain learn and maintain such precise connections as neurons die off and affected by biological noise (e.g., temperature) which has not be answered in this model.

Answer 24

The Hopfield (1982) associative memory network uses standard artificial neurons with no dynamics. The representation of the network is shown below which demonstrates all neurons are connected with each other and that neuron Si is connected to Sj with weight of wij. The assumption of the Hopfield associative memory network is that it assumes a fully connected network with symmetric connections (wij = wij). Symmetric connections means that weight going in one direction is the same as weight going in another direction. The properties of the Hopfield associative memory network is that it contains simple connectionist neurons, no dynamics, we impose the update schelude, a sign function as a transfer function and units can be active (Si = 1) or inactive (Si = -1). The sign function used as a transfer function means if a value is below 0 then set it to -1 and if a value is above 0 then set it to 1.

Answer 25

This means that if the sender and receiver neuron are both active then the sender likely contributed to making that receiver neuron fire! Thus, it strengthens the connections between the sender and receiver neuron; that is their weight increases.

Answer 26

In Hebbian learning, we can change the weights of synaptic connections mathematically in Hopfield Associative Network by taking one of the weight (wij) and add to the product of activity of pre and post synaptic neurons (if both active) and multiply by very tiny number which is epsilon. Epsilon is used in the equation as it is a very tiny number as we don’t want to change the weights in the network too quickly. Additionally, In most cases, you will want to incrementally learn something new (i.e., have multiple presentations of two stimuli together to associate them together).

Answer 27

The Hopfield Network learns by imposing a pattern we want to learn then letting the learning rule act. Imposing a pattern means that we clamp an activity of a subset of neurons for one pattern and let the Hebbian learning rule act to change the synaptic weight connections in the network by taking one of the weight between neurons and add to the product of activity of pre and post synaptic neurons (if both active) and multiply by a tiny number (epsilon). When imposing a pattern, if both neurons are not active (i.e., both -1) then the weight between these neurons increases and their connection is strengthened. If both neurons are active (i.e., both 1) then weight between these neurons are increased and connection between them is strengthened. However, if activity between two neurons are mixed (i.e., one is inactive [-1] and one is active [1]) then the weight goes down and may lead to pruning of the synaptic connection between these neurons. These pattern of activation is learned as stable states under the rules for updating activations. Stable states mean the update rule produces no more changes in the active neurons in network. Thus, when a pattern of activation does not change anymore, we say a stable state has been reached. The update rule can be asynchronously where one unit of the network is updated at a time (at random or pre-defined order) or synchronously where all untis are updated at the same time. In update rule, many patterns can be learned in same network but memory capacity is limited to ~0.14N (N is number of neurons in Hopfield network). The memory learned in Hopfield network is content addressable, can perform a pattern completion of a partial cue.

Answer 28

Content addressable simplify means that part of the content of the memory is sufficient to address to find the complete memory. This means the network can perform a pattern completion of a partial cue of completing pattern of activation of learning from partial input.

Answer 29

Say we have a memory of “I saw a magneta turtle that was squeaking”. This would tirgger the activity of the neurons in visual cortex that represented a magneta turtle as well as neurons in auditory cortex that represented the squeak sound. There will be direct connections of neurons in Hopfield associative network to other neurons. Pattern completion will continue in Hopfield associative memory store and also extend to reactive the neurons in the sensory cortices that was first active when you first memorised the thing (e.g., “the magenta turtle that was squeaking”).

Answer 30

This is because the hippocampus has an extensive connections to virtually ‘all association areas (polymodal) in neocortex. But there is no necessarily direct connections to early (unimodal) sensory cortices. So the sketch is a severe simplification.

Answer 31

They are discrete attractors as we would not want a continuous attractor as there will easily be interference between the different patterns of activation in network. Thus, Hopfield network memories are discrete attractors as we want to separate our memories.

Answer 32

Once learning is done (see paragraph of learning) it can perform recall which involves starting from a pattern similar to memorised pattern of activation, change activation according to sign of input (update until no changes occur) to recover the original pattern.

Answer 33

The Hopfield network learns by imposing a pattern activation we want to learn (i.e., the purple cow mooing) and then letting the learning rule act. Imposing a pattern means we clamp the activity of a subset of neurons for one pattern of the purple cow was mooing and let Hebbian learning rule act to change the synaptic weight connections in network. Hebbian learning does this by taking the weight between neurons and add it to the product of activity of pre and post synaptic neurons (if both active) and multiply by a tiny number (epsilon). When imposing the pattern, if both neurons are not active (i.e., both -1) then the weight between these neurons increases and their connection is strengthened. If both neurons are active (i.e., both 1) then weight between these neurons are increased and connection between them is strengthened. However, if activity between two neurons are mixed (i.e., one is inactive [-1] and one is active [1]) then the weight goes down and may lead to pruning of the synaptic connection between these neurons. The learning rule will act until there is no more changes in the set of active neurons

Answer 34

The Hopfield network does not work in isolation as there is direct connections of neurons in Hopfield associative network and other neurons. Such as the memorisation of seeing a purple cow that was mooing in Hopfield network also activates the activity of neurons in visual cortex that represent a purple cow as well as neurons in auditory cortex that represent the moo sound.

Answer 35

The memory learned in Hopfield network is content addressable, can perform a pattern completion of a partial cue which can recall the complete memory. Content addressable simplify means that part of the content of the memory is sufficient to address to find the complete memory. For recall, it involves to start with a pattern similar to memorised pattern of activation , change activation according to sign of input (update until no changes occur) to recover original pattern. To recall the purple cow mooing, we give a partial cue of a cow mooing which will cause two neurons to be active (auditory) and through pattern completion these two active neurons will drive for another neuron to be active which represents the visual representation of the cow; making the associative memory store complete. Thus, we can recall a nearby (i.e., similar) pattern that “I saw a purple cow that was mooing”. Pattern completion will continue in Hopfield associative memory store and also extend to reactive the neurons in the sensory cortices that was first active when you first memorised the thing (e.g., “the magenta turtle that was squeaking”).

Answer 36

Its from the cortex into DG then sends CA3, CA3 sends to CA1 and CA1 sends to subiculum

Answer 37

It is connections from EC to DG to CA3 to CA1.

Answer 38

In addition to being able to impose activity on CA3 and override its current connections, DG can also perform a function called pattern separation. Pattern separation is opposite of pattern completion. DG performs pattern separation in DG as if two incoming neural patterns are fairly similar (i.e., a large proportion of the same neurons are active at the same time), then the DG makes these patterns dis-similar. However, the overall function of DG is poorly understood since it does not fire often. DG also is location of adult neuro-genesis where new neurons are born.

Answer 39

association of information-rich (EC-CA1) and information poor (compressed) in CA3 > CA1 streams. Therefore, at time of encoding memory in CA3 since the memory is compressed when it goes to CA3 then you can retrieve a lot of detail of memory since ther is a direct connection from EC to CA1 hetero-association which produces more information rich representation which is then associated to CA3.

Answer 40

different information that enters the hippocampal circuit at different dorso-ventral level but CA3 extends across the entire longitudinal axis of the hippocampus and can associate the information at the different levels. For example, at one dorso-ventral level information about affective state may enter (e.g., fear). At another dorso-ventral level, spatial information may enter (e.g.. where was I in the environment) and CA3 synapses can associate the affective and spatial information together.

Answer 41

1) Perfornant path synapes in DG form new representations of input from EC 2) Mossy fibres from DG to CA3 induce a sparse pattern of activity for auto-associative storage 3) Excitatory recurrent connections in CA3 mediate auto-associative storage and recall of these patterns 4) Schaffer collaterals from region CA3 to CA1 meditate hetero-associative storage and recall of associations between activity patterns in CA3 and activity induced in CA1 by entorhinal input 5,6) Perfornant path inputs to region in CA1 form a new representations of entorhinal cortex input for comparisson/association with recall from CA3 7,8) The comparison of recall activity in region CA3 with direct input to region CA1 regulates cholinergic modulation. Mismatch between recall and input increases ACh, match decreases ACh 9) The theta rhythm may time encoding vs retrieval modes

Answer 42

since HM had intact memories of old childhood memories but memories from 5-10 years before lesion lost and forgot death of favourite uncle in 1950. HM retrograde amnesia. Retrograde amnesia is loss of memory access to events that occurred or information learned before an injury or the onset of disease. The temporal gradient of retrograde amnesia says recall for events immediately leading up to onset is poorly but earlier memories remin intact. The temporal gradient of retrograde amnesia show memories depend on hippocampus temporally and sugesst hippocampal memories are consolidated in neocortex over time and therefore become hippocampal independent.

Answer 43

There is fast and slow learning in memory consolidation in hippocampus. If we experience an event and memorise it then hippocampal recurrent collaterals for rapid (;one-shot) associative learning. Then we learn an event where hippocampus trains slow learning in neocortex and slowly update weights.

Answer 44

Form strong connections in CA3 to be able to pattern complete to cortex Overtime, the connections in hippocampus wil decay away once you established strong connections in cortex

Answer 45

Issues of fast and slow learning in memory consolidaiton How exactly is this rehearsal happening and how fast is this transfer? (One night or 20 years) model of consolidation of memory and how it happens is a hot topic and ongoing debate.

Answer 46

1. At a given location a small number of them is active 2. Using Place cells in CA3 and other cells outside hippocampus (e.g., code for affective state/intention) to Associate together (i.e., bind) the inputs related to an event that is available in from a given location 3. Provide a medium for pattern completion in CA3 4. At a given location by forming these connections, an attractor is formed across CA3,consisting of spatial and non-spatial information. 5.When position changes, a new set of place cells is active, and a new associations is retrieved

Answer 47

The first property of place cells is that their location of place fields can not be predicted based on anatomical vicinity. This means that anatomically close place cells have place fields far from each other (rather than close to each other) or one place cell can be silent while other is active in different environments.

Answer 48

The second place cell property is that: 1) firing is independent of rat’s orientation in open field, 2) firing is directional in narrow-armed mazes and, 3) firing is robust to the removal of sensory cues. Firing is independent of rat’s orientation in open field since activity of place cell is still in same location if animal direction west to east or east to west. Firing of place cells is directional in narrow-armed maze (linear track where animal runs back and forth) meaning the place cell fires in one direction. Research from Nakazawa shown firing of place cells is robust to removal of sensory cues as if you have 4 sensory cues in enviroment and remove ¾ of them then place cells still fire at specific locations of the enviroment. However, if you block the NMDA receptor, this is not the case, which means the formation of place cell fields is important for plasticity by NMDA-receptors for these place cells.

Answer 49

The third property of place cells is global remapping. Individual place cells at given environments fire at different location or are silent and together all active place cells likely to cover the entire environment with their place fields. Some place cells are silent in one environment but active in another. If a place cell is active in both environments then it is so at different locations. Global remapping is process of changing firing locations or turning on/off between environments

Answer 50

We can think of set of active place cells across environment akin to extended attractor network (current context recruits a given set). We need to be careful since a set of active place cells bounded together by CA3 recurrence it can not be via CA3 place cells with small receptive fields. Must be via CA3 cells that are active throughout the environment (i.e., not just CA3 place cells)

Answer 51

Kentros et al., (1998) showed that if you inject saline , place field is stable. Place field stability falls when NMDA receptors are blocked (i.e.,NMDAR anatgonist is injected in rat)

Answer 52

The sharp model of place cells uses a learning rule called competitive learning rule. There is a diagram of the competitive learning rule below which shows the following: 1) some input patterns, 2) some neurons will be active or not, 3) there will be an output neurons (i.e., receiver neurons), 4) initial random weights between each input and output neuron and, 5) strong inhibition between the output neurons (via lateral inhibition but don’t model them explicitly).

Answer 53

For input pattern X, there will be initial random connections and a particular output neuron (at random) Oi will be the most active neuron by setting it to 1 (maximum activity) and other Ok neurons to 0. After this, Hebbian learning will be conducted where wij => wij + epilson OiXkj and do this for all other connections where output neuron is active and update weights between neurons. We do not do this if activity of output neuron is 0 since it will not change the weight. Epilson is a very tiny number as we want small incremental changes to weights in Hebbian learning. The equation of Hebbian learning means the activity of given output and input neuron is multipled by each other and output is made very small by epilson and added to pre-existing weight (wij). The process of Hebbian learning is repeated with presentations of each input pattern. The result of competitive learning rule is that outputs whose incoming weights are most similar to pattern xn wins and its weight become more similar as we do learning.

Answer 54

Before updating the weights in Hebbian learning equation in competitive learning rule between neurons, we will need to normalise the synaptic connections first. Normalisation is for a given neuron, the total sum of connection (i.e., input) weights to each output neuron stays constant (e.g., sum [wi] = 1). The sum(wi) = … can be any random number. We keep track of normalisation as we divide the weights after each learning step by the sum of total weights (sum[wi]) so sum will always stay 1. For example, in this diagram sum of all inputs weights going onto output neuron 02 is always 1: Normalisation process repeats with each presentation of input pattern.

Answer 55

If we did not have normalisation then if we continue learning for a long time, the weight will become very strong in Hebbian learning. There must be a physiological limit in the brain as how strong a synaptic connection can be. The output neuron can only fire at a given maximum rate.

Answer 56

We keep track of normalisation as we divide the weights after each learning step by the sum of total weights (sum[wi]) so sum will always stay 1. For instance in diagram, weights after first learning step is 2,1,8,3 and 2 . Sum(wi) = 2 + 4 +5 +3 +2 = 16 => update weights to be 2/16, 1/16, 8/16, 3/16,2/16 so sum of all weights is still 1. The strongest weights will be stronger wij  wij epilson oixj. For example, wij = 8/16 grows stronger faster at expensive of wij = 1/16. The weakest weights get weaker.

Answer 57

Model proposes that maybe at given location, a certain sensory inputs are very strong and selects and map place cells on this. Maybe at different location, different sensory inputs are active and map those onto place cells. In this way, we get location specific input.

Answer 58

Sharp’s model of place cells was developed in 1991 and uses standard artificial neurons with no dynamics. There is a diagram of sharp’s model of place cell firing simulation shown below: There is a simulation of rat (blue triangle) that will move around the circular box and cues around the box (A,B,C etc…). At each location, the simulated rat will observe the distance to visual cues and “observes” direction of cues relative to itself So neural activity is propagated and perform learning updates. Then the simulated rat will move and explore the box and rotate a small distance. What happens to the neurons as simulated rat moves around and observes distance to cue and observes direction to cues in Sharp’s model is that there is two stages of competitive learning: 1) conjunction so distance and direction of landmarks is learned (i.e., input pattern is now a representation of distance and direction to all cues) and, 2) conjunctions of there first stage output yields place cells.

Answer 59

There is simulated place cell firing is resistant to cue removal in model as at each location , activity of PC output neurons is calculated and get these place fields for specific locations for cell 3, 9 and 13. . If you remove some cues, the place field is still there (exactly see experimental) as subset of remaining cues is sufficient once learned correct connections to reactive that cell. Stimulated place fields are omni-directional only after random exploration and not followed directed exploration. This is shown as we get directionality in linear tracks. Model recorded location and direction of “stimulated rat”.

Answer 60

O’Keefe and Burgess, 1996 investigated what gave inputs to place cells. In their method, they made a square environment where the rat forages in a box where the size of the box is varied across dimensions and performed single-cell recordings on place cells. They obtained single-cell recording data on place cells where they extended the box across 1 dimension and in another dimension and then extended the box on both dimensions. They found that some place cells they recorded exhibited ‘simple’ place field firing which is consistent across box sizes. However, some place cells appear to change their firing when the geometry of the box changed. For example, the firing field of place cells was close to east wall. When the wall extended on that dimension (towards east), the firing field of one place cell neuron changes and extends so there is place cell spikes all along the east side of the wall of the box. Thus, this showed that geometry of space is an important factor (at least for a subset of place cells).

Answer 61

After O’Keefe and Burgess showing geometry of space is certainly an important factor (at least for a subset of place cels), Hartley coined the term boundary vector cells (BVC) cells to describe cells that respond to a presence of a wall. BVC have not been found experimentally around the time of Hartley et al’s research so they did theoretical predictions. Hartley et al hypotehsised each BVC is tuned to respond when a barrier lies at a specific distance from the rat in a particular allocentric direction. In that BVC receptive field at given orientation in enviroment relative to enviroment. As the animal approaches the wall, the wall of a specific barrier/boundary of box, covers more of the receptive field and firing rate of a particulae BVC goes up and is maximally. These BVC cells are allocentric (world-centred) meaning the direction (of barrier) defined relative to some element of environment (like an external cue) and does not depend on animal’s orientation. Hartley et al. (2000) prpoposed that BVC firing is determined by product of two gaussian functions defining distance and angle tuning to boundary. There is shorter distances reflecting less uncertainty in inputs to BVCs. They said if you take a set of BVCs with receptive fields in different directions that are active, they together signal the distance to a set of boundaries

Answer 62

Hartley et al. (2000) theoretically predicted/hypothesised that boundary vector cells (BVC) may give (geometric) input to place cells (PC) and influence their activity. In other words, Hartley hypothesised we know how far we are from the wall (BVC) as we know where we are (PC). They created a BVC model of place cell firing in which it involves ¾ BVC which give input weight to a PC and you can sum BVC input activity * input weight and put through transfer function to produce a place field. When 3 BVC cells are active, these different BVC cells will be maximally active and respond to barriers at specific directions and distances. These three different active BVC cells give input to PC and is maximally active at this location. We can then subtract the background activity and get place fields at particular locations.

Answer 63

The BVC model of place cell firing model experimental data from O’Keefe and Burgress as they calculated the activity of place cells at all possible positions/locations (x,y) simultaneously since one location does not influence the other. We could do this location by location but nothing stops us at doing it at all locations. Hartley modelled O’Keefe and Burgress (1996) using 2-4 BVC oriented at right angles to one another and thresholds are sufficient to fit most (i.e., give best match) of PC fields in data by O’Keefe and Burgress (1996). They showed there was a close match of place fields between O’Keefe and Burgress data and Hartley’s model results.

Answer 64

1. Record some place cells in differerent shapes of enviroment (e.g., square, circle) 2. Find BVCs that best fit to those place cells in (e.g., square/circle enviroment) 3. Use the model BVCs to predict firing in other enviroments that have different shapes (e.g., right angle-triangle) 4. Experimentally, then record the same PC cells as found in the step 3 in predicted enviroments (e.g., right-angle triangle)

Answer 65

Discovered BVC in rodent subiculum (area close to hippocampus). Rodent has a receptive field for west and everytime rodent is near the west wall of a box, a cell will fire does not matter if curtain around box, remove the curtain or change shape of the enviroment, the specific BVC will always fire west to the wall

Answer 66

Discovered BVC in rodent subiculum (area close to hippocampus). Rodent has a receptive field for west and everytime rodent is near the west wall of a box, a cell will fire does not matter if curtain around box, remove the curtain or change shape of the enviroment, the specific BVC will always fire west to the wall

Answer 67

Discovered BVC in rodent subiculum (area close to hippocampus). Rodent has a receptive field for west and everytime rodent is near the west wall of a box, a cell will fire does not matter if curtain around box, remove the curtain or change shape of the enviroment, the specific BVC will always fire west to the wall

Answer 68

can not account for the long-term dynamics of place fields in similar environments nor the importance of synaptic plasticity in place field stability. Additionally it can not explain global remapping to place cells. Global remapping is process of changing firing locations or turning on/off between environments. Global remapping is akin to change between attractor but there is slow experience dependent changes to place fields. The attractor is slowly deformed over time (i.e., changing the active representation of one enviroment by moving to new enviroment). It is likely to reflect the synaptic plasticity and can not be explained by a ‘fixed’ model (i.e., BVC and sharp model) = leading to development of extended BVC model.

Answer 69

The extended BVC model uses the learning rule called Bienenstock-Cooper Monroe (BCM) rule and it is a modification of Hebbian plasticity. In extended BVC model we update weights using BCM rule where it required the pre and post synaptic activity (xi and yi). The weight reduces if post synaptic activity is low and weight increases if activity is high. In extended BVC model using BCM rule we change the weights and update the activity of yi (place cells) of summing the BVC input activity * input weights and put through transfer function. There is a sliding threshold in extended BVC model using BCM rule. The threshold of yi output firing is proportional to the square of recent activity of yi (place cells). The BCM rule xplains if you get a pattern of activity x drives neuron y then if y less than threshold then weight decreases (unlearn weak inputs) and if y more than threshold than weight increases (learning). We update the threshold based on average activity of y.

Answer 70

* From BVCs, and get PC field representation * Repeated so get PC field representation for all PC * Update the weights between BVC and PC using BCM rule * What happens if you have a weak secondary peak in your PC at a specific location in enviroment then learning rule will weaken the weights for the neuron that gives you this weak secondary peak over time

Answer 71

The network learns useful weights soley based on the statistics of input patterns

Answer 72

Classifying input patterns vis competitive learning (winner takes all) BCM rule (unlearning below threshold) Basic Hebbian learning rule

Answer 73

The network is given a desired output to compare its own output to The difference instructs the change in synaptic weights

Answer 74

Perceptron (i.e., delta rule) , deep learning models,

Answer 75

The network is given intermittent feedback in form of punishment and rewards and uses this as a guide to change weights.

Answer 76

depend on the brain area, the type of information that is learned, thestage of development of the organism etc.

Answer 77

- * Describes how a set of examples of stimuli and correct responses can be used to train an artificial neural network to respond correctly via changes in synaptic weight - Learning governed by the firing rates of the pre- and post-synaptic neurons and the correct post-synaptic firing rate (i.e.,a teaching signal) => “supervised learning” - Uses standard artificial neurons with no dynamics

Answer 78

important historical example: instructive in understanding the aim of using neural networks for pattern classification.

Answer 79

* One output neuron (O1) and two input neuron (x1 and x2) * X1 and X2 neuron each have a input weight of w1 = 1 and w2 = 1 * To get output (activity of O1 neuron), you sum the input activity * input weight and put through trasnfer function (which is step function) * Output neuron is active if both input neurons are active (given they reach a threshold of 1.5) = model performs logical 'AND' function It is only when two input neurons are active that the output neuron is active

Answer 80

You can produce a graph of all of the possible x1 and x2 combinations from simple perceptron model where dashed line is decision boundary and left of line is where 01 = 0 and right of line is 01 = 1. We can write equation of line: x1 +x2 = 1.5, x2 = -x1 + 1.5 Then line separating o1 = 1 and o1 = 0 is where net input equals the threshold which is w1X1 + w2X2 = T We can rearrange threshold equation to make equation of line which is: x2 = -(w1/w2)x1 + T/w2 (format of y = mx + c and implicility assumed W1 = W2 = 1). Changing the weights will change the equation of line Learning is changing the weights to classify two input patterns means finding weights so line separates group of 01 = 0 and 01 = 0. An input pattern is simply a set of activity patterns across input neurons.

Answer 81

* X1 signals weight * X2 signals height * Each individual (child/adult) we test will be one combination of two input neuron (weight/height) = pattern * We train/learn the network on many example patterns of individuals, each individual will change input neuron's weights and place a decision boundary to separate two groups (height and weigt) * Decision boundary can be fuzzy since there may be kids tall or heavy and adults small and light but overall classifer separate children and adult based on height and weight * After training this network on 100 individuals (50 adults/50 children), if we present 101's person (that has not been used to train the network) , then we measure performance on how well it generalises (i.e., how well classifies the 101's individual correctly):

Answer 82

Learning or training in perceptron model means we present example patterns and each example pattern changes the input weight. The training set is set of patterns across neurons (e.g., x1 and x2 representing weight and height which is the categories we want to classify adults and children with). Performance of pattern classification in perceptron depends on generalisation which is giving a new, never before seen datapoint (X1, X2) and observe the output (result of classification. Performance of perceptron is how well the perceptron neural network classifies when presenting new datapoints.

Answer 83

This is done through the delta rule which is an example of supervised learning rule. We will use supervised learning rule on a complex model of perceptron which has: 1) 5 input neurons X, 2) 3 output neurons (i.e., 3 possible results of pattern classification) , 3) start with a initial random weights, 4) outputs are n = 3 (o1k, o2k, o3k), 5) input is n = 5 (x1k, x2k, x3k, x4k, x5k). K means pattern K and output K which is elements of training set (i.e., set of patterns across neurons).

Answer 84

In 3 output and 5 input neurons, which we have set of patterns across neurons (i.e., our training set). Present a pattern k and find a corresponding output values (o1, o2k...). So present pattern k and each output neuron perform computation as sum inputs activity * weight and put through transfer function and see if output is on or off based on inital random weights. We know if network has classified pattern k correctly by using delta rule in which has teaching signal for neural network (i.e., has acess correct target output for pattern k). In delta rule, change in connection weights performed according to difference between the target and the output we get for the first input pattern k. Then represent next input pattern k --> k +1 and update weights again. The delta learning rule in perceptron model changes input weight to reduce the error. The change in connection weight is big if discrepancy between target and output activity is big.

Answer 85

Specifically, looking at connection weight from x1 and output neuron 02 Output activity of 02k , after computation, is o2k= **0.2,** Input activity of x1k was 0.7 We also have target t2k is: **0.3,** so output of activity o2 want 0.3 instead of 0.2 The delta learning rule takes into account by changing connection weight between x1 and 02 by calculating: w12 (connection weight of x1 and 02) + epilson 0.7 (0.3-0.2) so the error is reduced!

Answer 86

* artifical retina of inputs x * Single output neuron that turn on or off depending whether pattern on artifical retina was correct one * Single output neuron on if pattern detected (using transfer function) * Presented input patterns xk * Some patterns detected with target (tk=1) and some are not (they are foils) * Each pattern , apply the delta learning rule * After many presentations of whole traning set (in random order) it would find the best linear discriminator of target from foils * Note the connection weight only change if there is an error and if input neuron is active * If target is bigger than output then weight increases * If target is lesser than output then weight decreases * This delta rule changes the weights to reduce the error

Answer 87

Minsky and Papert (1969) said that the perceptron can perform linear discrimination but can not solve ‘non-linearly separable problems’ A non-linear separable problem , for example, can be shown below where there is a graph of possible combinations of input x1 and x2. Say you want to learn x1 or x2. The output neuron would be ON if X2 = 1 and X1=0 OR X2 = 0 and X1 = 1. Can we place such a line that divides the space such as output is 1 and output is 0 on other side? We can not do this separation linearly. Rosenblatt’s Perceptron performs linear discrimination where decision boundary always linear from combinations of input neurons where either side output neuron is active or not (i.e., 1 or 0). Therefore, perceptron can not solve non-linearly separable problems as it always uses a linear decision boundary. However, if we include hidden layers in perceptron which gives it more power then able to solve these ‘non-linearly separable problems’.

Answer 88

Reinforcement learning deals with the question of how to make a decision in sequential problem like chess or solving a maze. This is difficult because of the temporal credit assignment problem as you don’t know which of your past actions was pivotal for a good outcome (e.g., winning a game of chess).

Answer 89

agent in the environment that receives reward intermittently. The rewards are positive or negative but not otherwise specific as in supervised learning. But the agent does not know the rules of the environment meaning it does not have the model of its environment. The artificial agent has policy, state and reward. Agent has a policy which is a set of rules that determine which actions to take for different states of myself in environment such that I (hopefully) maximise my future reward. Reward R is where the environment provide reward which affects state of agent. For a given policy, a value V can be associated in given state which is sum of expected future reward for each possible state. At any moment the agent is at a given state. V is calculated by formula which has a discount factor which means further in future you expect the reward less likely you value it.

Answer 90

The goal of reinforcement learning (especially model-free reinforcement learning) is to optimise the policy to maximise future reward. This is a hard problem as in a simple game of tic tac toe we can write down all the states but for chess it is astronomical possibilities of states and can’t write them all out.

Answer 91

At any given moment the robot is in a given state (i.e., distance and direction to the ball which characterises its mental repertoire) and can take a set of possible actions (move up/move down/move left/move right/kick ball). When robot will kick the ball it gets a reward (e.g., a number in memory it wants to maximise) and up until then it gets nothing.

Answer 92

The learning algorithm guides robot learning which says it wants to maximise reward. According to learning algorithm, robot has learning episodes where it evaluates its own state, take an action and observe reward (robot may get reward or not) and these episodes last until ball is kicked and reward is received. if robot receives reward something in environment has been learned. If it has not then robot takes another action.

Answer 93

The football kicking robot does not know which actions lead to reward and performs actions initially and only learns at the end of learning episodes. At its first learning episodes, it performs random actions and kicks when there is no ball in the field. But at some random point in future the robot manages to kick the ball and receives the ball. It has now learned when it is zero steps from the ball it kicks it. However, it has not learned when it is one step away from the ball to kick it and only learn in next learning episode. Its only learned the last state-action pair.

Answer 94

In Q-learning, we can keep track of progress of robot learning to kick ball in Q-table which mimics outline of environment and assigns value for each action. For example, here is kick table for first learning episode. Value is performing action (A) given state (S) and thought of as current prediction of how much reward robot will eventually obtain if given state S it performs action A and subsequent high value actions. In second learning episode, this time the robot reached the ball above and find ourselves in field with ball and know how to kick because of Q-table for KICK. Can assign a different value to state-action pair that led us to the field with ball to kick by assigning value of 8 (Smaller than 10 in KICK table as once step away from reward) in B7 in move-down Q-table which was zero prior to learning.

Answer 95

At each learning episodes there is then partial values assigned immediately preceding state-action pair if one steps into field with previous learned value. The trial and error part of the robot reduces across Q-learning episodes and build more Q-tables which assigns values to all the states of robot of different actions. Thus, this leads to route of ball to kick. After robot learned the route to ball to kick, it can look up the actions in Q-table and performed a learned action sequence as does not need to rely on trial and error anymore.

Answer 96

Q tells us ‘joint quality’ of taking an action A, given a state S. The formula of Q means probability of entering state S’ (next state , given S, A) * total reward. So the probability of transition of state times reward. In The formula of Q means transitioning from current state to next state is not completely deterministic

Answer 97

It is a model-free, off-policy reinforcement learning that will find the best course of action, given the current state of the agent.

Answer 98

Can learn complex behaviours No need for an explicit teaching signal as in supervised learning and only intermittent reward

Answer 99

Takes time, lot’s of trial an error. Especially if the state-space (number of possible state-action pairs) is large (e.g., as in chess) But.. But we can approximate the Q-table, approximate it with deep learning Deep Q learning: e.g., Deepminds Atari game playing AI uses a DQN (Deep Q Network) Cannot apply in all types of situations: Can’t fall randomly off a million cliffs to learn optimal behaviour. Sounds funny until you consider self-driving cars! Training in the real-world (e.g., a robot) would take a long tim

Answer 100

Temporal difference learning is a model-free reinforcement learning (RL) and slightly different from Q-learning. It links classical conditioning and reinforcement learning together in order to overcome the temporal credit assignment problems that occurs in Rescorla-Wagner model.

Answer 101

In temporal difference learning model, we will need the value function V at time t (Vt) to predict the sum of future rewards, not just immediate rewards so we can learn S2 predicts S2 which predicts R (reinforcement = general like food reward/punishment) In other words, we want the sum of all rewards at times tau greater than now (present at time t). We want to decompose V(t) to current and estimate of subsequent reward. To do this, we need a delta rule to ensure this happens. Delta at time t is current reward + future reward – expected reinforcement (what I expect) so delta becomed the difference between expected reward now and estimate of all future reward.

Answer 102

Q-learning update formula and maps onto it well TD-0 first , 0-rder is closet to Q-learning.

Answer 103

We can introduce further time steps in future and discount them using discount factor = as further way in future uncertainity increases We can extend TD learning to include more states in future e.g., 2 time steps in i.e., - (2) = similar to Q-learning we can estimate two steps into future and if reward is to be expected we update our V (current estimate value of expected reinforcement) Same form of update as we had in Q-learning expect it is not on state-action pair updating

Answer 104

Delta is based on current reward only as its delta is the difference between current reward (reinforcement) and expected reinforcement given all stimuli

Answer 105

The Rescorla-Wagner rule attempts to describe the change in associative strength (V) between a signal (conditioned stimulus, CS) and a subsequent stimulus (unconditioned stimulus, UCS) as a result of conditioning trial. Classical conditioning can be modelled with this rule.

Answer 106

The two conditioned stimuli (S) that can predict reinforcement can be seen in figure below in which if the reinforcement neuron is active then we predict reinforcement (could be reward/punishment). The way in which two stimuli that predict reinforcement can be modelled is implied by blocking and overshadowing experiments research conducted by Kamin in 1969 demonstrating not all stimuli present during learning subsequently control behaviour. In a blocking experiment he did, a dog is repeatedly exposed to a tone (first conditioned stimulus, CS1) together with food (unconditioned stimulus, US). The dog salivates when the tone is presented (conditioned response, CR). After several consecutive conditioning trials, this time the tone (CS1) and light (CS2) together with US , causes dog to not salivate / has weak response to light CS2) when tested separately later. The stimulus control by CS2 has been blocked by earlier pairing of CS1 with US. In an overshadowing experiment, a light CS1 is more salient than a tone CS2, then effect of pairing a UCS with light + tone compound will be strongly associate the light with UCS (food), with little associative strength developing between the tone and UCS (the light overshadows the tone); this causes no response (i.e., salivation) to CS2.

Answer 107

These blocking and overshadowing gave answer that way in which two CS predicts R can be modelled is via the Rescorla-Wagner rule. The Rescorla-Wagner rule attempts to describe the changes in associative strength between a conditioned stimulus and a unconditioned stimulus as a result of conditioning trial. This rule has a delta rule of taking the difference between reinforcement (general term which could be reward or punishment) and produces a single error term for all stimuli.

Answer 108

Rescorla-Wagner rule can not be applied when there is a temporal sequence of stimuli that occurs in second-order conditioning. For instance, second-order conditioning can be demonstrated using the following procedure: a CS1 (e.g., a light) is paired with a UCS (e.g., food) in phase 1; then CS2 (e.g., a tone) is paired with CS1 (the light) in phase 2. Tested response to S2. This resulted in a CR relevant to the original UCS (food) being evoked by CS2, even though CS2 has never been directly paired with food (e.g., Rescorla, 1980; Rizley & Rescorla, 1972). The Rescorla-Wagner rule can not be applied to explain why there was a response to CS2 since it only works for direct associations of a CS with reinforcement (R; food reward/punishment). More specifically, the rule does not work since there is no R in phase 2 of second order conditioning and its simple delta rule it uses depends on R! This is because the simple delta rule it uses is calculated by taking the difference between received and expected reward.

Answer 109

During recall: If the EC->CA1 connection generates one pattern in CA1, but the CA3->CA1 connection generates a slightly different pattern from memory (via pattern completion in CA3), then the mismatch can be an indicator of changes in the environment.