14. Interpreting the Theory

Question 1

Wolfram Schults (1997, 1998)

Answer 1

• midbrain dopamine release represents a reward prediction
• rewarding stimuli have 3 basic functions: (2 important)
  1. rewards elicit approach and consummatory behaviour and serve as goals of voluntary behaviour
• interpret ongoing behaviour, change priorities of behavioural actions, needs to bias future behaviour

2. rewards have positive reinforcing effects and this increases the frequency and intensity of behaviour leading to such objects
3. rewards induce sub subjective feelings of pleasure (hedonia) and positive emotional states

Question 2

2. rewards have positive reinforcing effects and this increases the frequency and intensity of behaviour leading to such objects

Answer 2

learning happens when rewards occur unpredictably and slows as a reward becomes more predicted

• reward driven learning depends on the “error:” between prediction of the reward and its occurrence (“coming back for more”)

Question 3

Raster Plots

Answer 3

• represents the firing of neurons in time
• horizontal row = trial number (stimulus presentation)
• each dot/bar represents a neuron firing
• trials are aligned with stimulus onset
• electrodes placed at level of the VTA (DA neurons firing) extracting AP’s

Question 4

learning

Answer 4

= comparing received reward with expected reward

• if an action produces an unpredicted negative outcome, it is unlikely to be repeated
• no further learning takes place when a reward is predicted by sensory cues
• fits both operant and classical conditioning
• but is this reflected in DA neurons?

Question 5

extracellular electrophysiological activity in monkeys

Answer 5

Schults et al., (1997) = raster plots

• recorded from midbrain DA neurons (VTA and Substantia nigra [SN]), recording extracellular activity whilst monkeys performed behavioural tasks
• naive monkey required to touch lever following presentation of a light
• touching the lever delivers juice
• after a few days, monkey learns to reach for lever as soon as the light comes on
• used raster plots in VTA (to measure DA neurons AP’s)

Question 6

Schultz et al (1997) - before learning

Answer 6

• drop of liquid occurs which isn’t predicted
• reward prediction error = DA neuron shows activation (in VTA) in raster plots, seeing a spike in DA firing after reward

Question 7

Schultz et al (1997) - after learning

Answer 7

• conditional stimulus predicts a reward
• no reward prediction error so DA neuron fires after the reward-predicting stimulus (CS = light)
• DA neurons do not fire after the presentation of a reward (juice)

Question 8

Schultz et al (1997) - after learning (no reward)

Answer 8

• there is tonic firing of DA neurons
• if the reward fails to occur there is a negative reward prediction error
• DA neurons fire after CS (reward predicting stimulus - light)
• but are depressed at the time of reward would have been given

Question 9

dopamine response =

Answer 9

reward occurred - reward predicted

Question 10

Schultz et al (1997) results

Answer 10

• the reward prediction error (RPE) acts as a teaching signal that is used to correct inaccurate predictions
• unpredicted reward (or something thats better than expected) = positive prediction = strengthen
• presentation as predicted = no new learning (DA fires after light - reward predicting stimulation)
• omission of predicted outcome = negative prediction = extinction of behaviour

Question 11

RPE and Blocking

Answer 11

• conditioning is impaired if a CS is presented together with a second CS that has already been associated with the response
• CS1 = fully predicts arrival of the reward
• CS2 = no change in RPE error = no learning
• blocked stimuli does not become associated with reward but unblocked stimuli does
• when an association as been made between reward and CS, when a stimulus isn’t presented there is a depression in DA firing, when there is a reward there is no DA response after reward given
• RPE = DA release = Learning (same thing when there is no reward)
• Enomoto et al (2011)

Question 12

Enomoto et al (2011)

Answer 12

• stepwise transfer
• if multiple actions are needed for the reward the DA response suggests that each bit is learnt at a time
• mid brain neurons transfer the right response to other stimuli (e.g. CS2)

Question 13

RPE and risk taking

Answer 13

• many naturalistic rewards have elements of risk

- Stauffer et al (2014) (gambling)

Question 14

Stauffer et al (2014)

Answer 14

• risk in lab measured with binary gambles
• monkey cued to make reward based decision
  1. 50:50 gamble large:small amount of juice
  2. guaranteed smaller amount of juice
• when risks are low monkeys gamble for bug rewards, when risks are high they take small rewards
• midbrain DA response the same, succecful gambles produce more positive RPE
• higher risk gambles are associated with:
  1.
  2.
• evolutionary advantage to gamble when risk low but not when high
• risk management = reduction encoded by reduction of DA activity after high risk gamble

Question 15

what are higher risk gambles associated with

Answer 15

1. high midbrain DA activity associated with cue

2. REDUCED midbrain activity at time of reward

Question 16

multi phasic DA response (concerns)

Answer 16

• timing of response = midbrain DA response too quick to carry any meaningful info about stimulus
• recent evidence shows that the midbrain response has 2 phases (Fiorillo et al., 2013)

Question 17

Fiorillo et al (2013)

Answer 17

• midbrain DA response has 2 phases
  1. an unselective transient phase that cues for potential rewarding stimuli
  2. second stage that codes for reward value of stimulus

Question 18

Does DA release at the NAc represent RPE?

Answer 18

• YES (Hart et al., 2014)

- fast scan cyclic voltammetry (like microdialysis but quicker)

Question 19

Hart et al (2014)

Answer 19

• fast scan cyclic voltammetry measured reward evoked DA release in NAc
• DA concentration in NAc displays a bidirectional RPE signal
• • symmetrical encoding of positive and negative RPE’s
• • DA release increased with positive error
• • DA release decreased with negative error
• more unexpected the lack of reward is the lower the DA concentration
• same as gambling/reward paradigm = more surprise + more reward = more dopamine

Question 20

implications of RPE processing in the NAc

Answer 20

1. conditioned responses - facilitates plasticity

2. action selection in basal ganglia

Question 21

1. conditioned responses - facilitates plasticity

Answer 21

• NAc integrates inout from cortical and sub cortical structures, often synapsing on same neuron as VTA sites
• coincident cortical NAc activity (synapse) with VTA DA activity strengthens the response
• DA release required for corticostriatal plasticity (Lerner and Kreitzer, 2011) - rewarding stimuli thought to be DA input that facilitates plasticity
• positive RPE = higher DA release = likelihood of potentiation (increase strength of nerve impulses along pathway)

Question 22

2. action selection in basal ganglia

Answer 22

• NAc projects to basal ganglia to initiate movement in pursuit of reward
• motor programs initiated basal ganglia on strongest drive for activity at any time
• when we have a positive RPE, a stronger bid is made by the basal ganglia

= movement more likely to be initiated is RPE larger

Question 23

Lerner and Kreitzer, (2011)

Answer 23

• DA release required for corticostriatal plasticity

- Rewarding stimuli thought to be DA input that facilitates plasticity