Week 6 (Chapter 47, 48) Flashcards
(39 cards)
What is thought to be at the centre of reinforcement learning in the mammalian brain?
The midbrain dopaminergic system
What are the uses for a theoretical framework for reinforcement learning?
- aid in the interpretation of neurophysiology data
- guide the design of future studies
The Rescorla-Wagner model formalized the idea that _______ is needed to drive learning
An error between the actual and predicted outcome
What is the learning theory originally formulated for computer learning?
Temporal difference (TD) learning
Reward expectation reduces dopamine reward responses in a purely _____ fashion
Subtractive
A classic adaptation of TD learning to a biological circuit model utilizes a _____
Complete serial compound (CSC) feature representation
Dopamine neurons show a negative prediction error ______
At the time of an expected reward
Dopamine responses to reward are suppressed only at _____
The time of the expected reward
What happens when the timing of a reward is shifted?
A larger dopamine response
Cues followed by late rewards result in _____ dopamine responses than cues followed by early rewards
smaller
What TD model was proposed to explain the shortcomings of the CSC TD model?
The microstimulus model
How does the microstimulus TD model differ from the CDC TD model?
The microstimulus model is able to account for the longer dip of dopamine responses upon reward omission
In 1998, Hollerman & Shultz discovered that when monkeys were given a reward stimulus earlier than expected, ______
There was a large dopamine response upon the reward, but no negative response at the time of the usual reward
What is one possible modification to the CSC TD model after Hollerman & Shultz’s discovery?
An animal is in two states - ISI when expecting a reward and ITI when not. When one is activated, the other is deactivated
Semi-Markov dynamics imply that time spent in a state is _____ and is defined by ______
- probabilistic
- a probability distribution called a ‘dwell time distribution’
The ______ model accounts for Hollerman & Shultz’s findings
Belief-state TD model
According to the belief-state TD model, uncertainty should ______
Dramatically affect how reward expectation evolves over time
The CSC TD is model-____, while the belief-state TD is model-____
- Free
- Based
What is a shortcoming of the belief-state model?
If an animal is hungry, food-based rewards would have higher state values than drink-based rewards - this is not explicitly learned, therefore the belief-state model does not account for it
Dopamine neurons code for ______
Reward prediction error
The magnitude of dopamine prediction error responses scales _____, integrating them into a biological teaching signal for ______
- reward size, probability, and delay
- utility
Rewards drive learning as _____
positive reinforcement
What brain areas are specialized for processing rewards and reward-related behaviours?
- dopaminergic midbrain
- orbitofrontal cortex
- amygdala
- ventral striatum
Dopamine neurons reside in the _____ and send signals to the ____ and _____
- midbrain
- basal ganglia
- frontal cortex
