Topic 5: Temporal Difference Learning

Question 1

Temporal difference learning

Answer 1

Change in environment over time causes change in expectations, which causes change in behaviour.

Question 2

TDL model vs. R-W model

Answer 2

1. Multiple vs. one step
2. Continuous vs. discrete
3. Temporal vs. per trial
4. Changes in expectation of reward vs. reward

Question 3

Dopamine codes prediction errors

Answer 3

Neurons that release dopamine appear to mimic the error function from temporal difference learning.

Question 4

Reward coding

Answer 4

Dopamine neural response is proportional to R(pS)

Question 5

Delay coding

Answer 5

The greater the delay from cue onset, the lower the intensity of dopamine neurons. When reward is provided, dopamine neurons fire.