Biological 5: Goal-directed and habit-based behaviour

Question 1

What is general vs specific PIT?

Answer 1

General PIT = stimulus paired with rewarding thing
Specific PIT = stimuli both paired with one specific reward

Question 2

What is a positive and negative prediction error?

Answer 2

Positive prediction error - increase connection of stimulus response
Negative prediction error - association between cue and outcome is weakened

Question 3

What does the Kamin blocking effect show?

Answer 3

Shows that prediction error is more important than things occurring close in time

Question 4

What two things can behaviour be?

Answer 4

1) Purposeful, ‘goal-directed’ actions
2) Automatic, ‘habitual’ responses

Question 5

What is the goal-directed system of behaviour vs the habit system of behaviour?

Answer 5

Goal-directed:
Stimulus -> outcome -> response

Habit:
Stimulus -> response

Question 6

Do animals learn my problem solving or trial and error? (Thorndike’s puzzle box)

Answer 6

Trial and error

Thorndike’s puzzle shows this:
When a cat figured out how to escape from puzzle box, time it took to escape gradually decreased as trials went on
If it had solved by problem solving, there should have been a sharp drop - a lightbulb moment
The shape of animal learning curve suggests that learning is an incremental process, reflecting the strength of association between stimulus and response

Question 7

What is Thorndike’s law of effect?

Answer 7

‘If in the presence of a stimulus a response is followed by a satisfying state of affairs, the connection between the stimulus and the response will be strengthened. If the response if followed by an annoying state of affairs, the connection between the stimulus and response will be weakened.’

Stimulus response association is mediated by outcome

Question 8

Does outcome drive conditioned behaviour or habitual responses?

Answer 8

Conditioned behaviour
Habitual responses not based on outcome

Question 9

What does a stimulus-response association result in?

Answer 9

Results in responding without any expectation that the response leads to the outcome (habit based responding)

Question 10

What does a response-outcome association result in?

Answer 10

Results in responding because there is the expectation that the response will result in the outcome (goal-directed responding)

Question 11

How can you tell goal-directed behaviour from habitual behaviour using the outcome devaluation test?

Answer 11

If behaviour persists despite outcome being lower in value = habitual
If it does not = goal directed

Mouse press lever to get cheese
Then devalue cheese - huge block of cheese - too full - cheese is less appealing
If behaviour is goal directed, then they do not press lever, because they are aware that pressing the lever leads to cheese, and right now they do not want the cheese
If behaviour is habitual, then they do press the lever because they do not know/are not concerned with the outcome

Question 12

What is Baxter and Murray’s (2002) outcome devaluation test on monkeys?

Answer 12

Training task 1: monkeys trained so that they get cherry for choosing correct object, and nothing for choosing incorrect object
Training task 2: monkeys trained on two different objects - gets peanuts for choosing correct object and nothing for choosing incorrect object.
Prior to the test the monkey is allowed to consume one of the rewards (cherries or peanuts) until sated. In test monkey is allowed to choose one of the two correct objects.
Which one will the monkey choose?
If they avoid the one that they are full of - behaviour is goal-directed

Question 13

If an animal is trained that lever 1 results in sucrose, and lever 2 results in pellets, and then sucrose is paired with LiCl (makes them ill), in the test phase, which lever should they press in behaviour is goal-directed and not habit based?

Answer 13

Lever 2

Same would happen vice versa if you devalued the pellets

Instrumental learning is generally the result of an outcome - goal directed

Question 14

If an animal is given extended training that lever 1 results in sucrose, and lever 2 results in pellets, and then sucrose is paired with LiCl (makes them ill), in the test phase, which lever should they press?

Answer 14

Levers 1 and 2 equally - behaviour is habit based not goal directed

Small number of training sessions - goal directed
Extended training - habits

Question 15

When is instrumental behaviour sensitive to outcome devaluation?

Answer 15

If it is still in early phases of training

• Instrumental behaviour is initially sensitive to outcome devaluation (i.e., it is goal-directed, R-O)
• If the instrumental behaviour is overtrained it is no longer sensitive to outcome devaluation (i.e. it is habitual, R-R)

Question 16

In what mental health condition do patients show an insensitivity to outcome devaluation?

Answer 16

OCD - more habit responding behaviour - not based on outcome

Question 17

Does stress result in less sensitivity to outcome devaluation?

Answer 17

Yes

Question 18

What substance can make someone less sensitive to outcome devaluation?

Answer 18

Alcohol

Actions learnt under the influence of alcohol are less sensitive to outcome devaluation

Alcohol seeking behaviour becomes insensitive to devaluation if over-trained

Question 19

What is the dorsomedial striatum involved in?

Answer 19

Goal-directed behaviour

Question 20

What is the basolateral amygdala involved in?

Answer 20

Incentive value

Question 21

What is the medial PFC involved in? Infralimbic vs prelimbic

Answer 21

Infralimbic - Encoding associations
Prelimbic - SR learning

Question 22

What are the four main parts of the dorsomedial striatum?

Answer 22

Caudate
Putamen
Nucleus accumbens
Amygdala

Question 23

Yin et al. (2005) conducted a devaluation test on rats who had lesions to the posterior dorsomedial striatum.
(left lever = sucrose, right lever = pellets, they eat one to satiety, which lever will they push?)

If DMS involved in habitual learning, there will be goal-directed behaviour
If DMS involved in goal-directed learning, there will be habitual behaviour

What did they do?

Answer 23

The sham rats pressed the lever much more when the outcome was NOT devalued

However the DMS lesioned rats showed no difference in lever pressing irrespective of whether the outcome had been devalued.
This is habit-based behaviour.

This shows that the DMS is important for goal-directed learning

Question 24

What is an issue with devaluation tests?

Answer 24

If an animal is impaired on the extinction test (choice after one outcome is devalued) it may be because they cannot discriminate between the rewards

Expectation of reward drives goal-directed behaviour - need a test that is more specific in showing response to expectation of reward

Question 25

What is the extinction test vs the rewarded test?

Answer 25

Extinction test: pressing lever leads to no reward Any change in lever pressing caused by the devaluation test can be only due to the expectation and knowledge of the outcome (therefore, R-O learning) Rewarded test: different levers lead to different rewards, and one is devalued Any change in lever pressing may be due to the impact of the now devalued outcome, during the test phase, on stimulus-response (SR) learning. For example, the stimulus-response is weakened by the now ‘non-satisfactory’ outcome. Need to be able to tell the difference between rewards to devalue a reward If they cannot tell the difference, behaviour towards both rewards will be the same Extinction test relies on expectation of outcome Reward test relies on knowing difference between rewards

Question 26

How do rats with posterior dorsomedial striatum lesions do on the extinction test vs rewarded test?

Answer 26

Extinction test: Right lever - nothing Left lever - nothing DMS lesion group failed this Rewarded test: Left lever - sucrose Right lever - pellets DMS lesion group passed this

Question 27

Why do DMS lesioned rats pass the rewarded test but fail the extinction test?

Answer 27

- The dorsomedial striatum is necessary for goal-directed behaviour (response-outcome learning). - Rats that received inactivation were not sensitive to outcome devaluation in the extinction test, they behaved in a habitual (stimulus-response) manner - Importantly, their lack of sensitivity to the outcome-devaluation procedure was not due to a failure to discriminate between the foods: they stopped responding to the devalued lever in the rewarded test when the devalued reward was actually presented - DMS does not play a role in identifying the difference between the two rewards DMS - important for expectation of outcome

Question 28

What is the role of the posterior dorsomedial striatum in learning?

Answer 28

Important for expectation of outcome

Question 29

What is the function of the basolateral amygdala in learning?

Answer 29

- Representing the sensory-specific properties of an outcome (and use them to discriminate between rewarded/non-rewarded outcomes) - Not necessary for acquiring instrumental responding, suggesting that S-R learning occurs independent of BLA function - Is necessary for knowing the current incentive value of the outcome, (e.g. sucrose more valuable than pellets)

Question 30

How do rats with BLA lesions perform on: Chain pull to get sucrose Lever press to get pellet Devaluation test for one of these

Answer 30

BLA - press devalued and valued similarly BLA necessary for the outcome representation, and using sensory specific information to learn which actions are rewarded and which are not BLA necessary for representation of the outcome

Question 31

How do control rats, BLA lesioned rats and DMS lesioned rats do on extinction test and reward test?

Answer 31

Control - can do both BLA lesioned - fail both DMS lesioned - fail extinction, pass reward

Question 32

What two areas make up the medial prefrontal cortex?

Answer 32

Infralimbic cortex Prelimbic cortex These areas have dissociable roles in goal-directed behaviour

Question 33

When infralimbic lesioned rats are given a low vs high amount of training and then do an outcome devaluation test, how do they perform? What is this area involved in?

Answer 33

Low training - goal-directed behaviour (press the valued more than devalued) - same as controls High training - persist in being goal-directed - different from controls - they turn to habit-based IL - involved in habit (as when it is lesioned, habit-based responding is inhibited)

Question 34

When prelimbic lesioned rats are given a low vs high amount of training and then do an outcome devaluation test, how do they perform?

Answer 34

Low training - habit-based responding - different from controls whose behaviour is goal-directed High training - persist in habitual responding - same as controls PL - involved in goal-direction (as when it is lesioned, goal-directed behaviour is inhibited)

Question 35

How do controls, infralimbic lesioned and prelimbic lesioned rats do on the outcome devaluation test with low and high amounts of training?

Answer 35

Sham - pass with low, fail with high IL - pass both (always goal-directed) PL - fail both (always habit based)

Question 36

Damage to the prelimbic region of the prefrontal cortex impairs sensitivity to outcome devaluation. However, damage after acquisition of the behaviour does not impair outcome devaluation. What might the prelimbic region be important for?

Answer 36

The prelimbic region is important for encoding associations, but is not where the memory is stored

Question 37

Damage to the infralimbic region of the prefrontal cortex impairs the development of habits. Inactivation after overtraining still results in habitual behaviour becoming goal-directed. What might the infralimbic region be important for?

Answer 37

Suppressing the influence of S-O-R associations on behaviour NOT important for learning S-R associations

Question 38

What are the prelimbic and infralimbic cortices roles throughout limited to extended training on S-O-R associations?

Answer 38

Limited training: The PL is necessary for initially learning R-O associations - goal-directed behaviour, sensitive to outcome devaluation Extended training: S-R associations are also acquired - behaviour becomes habit-based and insensitive to outcome devaluation, because the IL blocks R-O association If the IL region is removed after extended training, R-O associations are no longer suppressed, and behaviour becomes goal (R-O association) driven again

Question 39

Is behaviour dependent on both R-O associations and S-R associations?

Answer 39

Yes The amount of experience determines whether behaviour is determined primarily by one or the other learning system

Question 40

When behaviour becomes habitual, is it because the R-O association has been lost?

Answer 40

No Habitual behaviour occurs because goal-directed behaviour has been suppressed by the IL region

Question 41

What are NDMA receptors the basis of?

Answer 41

Associative LTM

Question 42

What is impaired in GluA1 knockout mice?

Answer 42

Reduced number and function of AMPA receptors Impaired synaptic plasticity (early, rapid form of LTP) Impaired short-term memory (short-term habituation - really recent memory of seeing or experiencing something) Intact associative long-term memory

Question 43

Can GluA1 knockout mice navigate a water maze?

Answer 43

Yes

Question 44

Can GluA1 knockout mice do a Y maze novelty preference task? (in which animals should preferentially explore a novel arm that they have not exploed)

Answer 44

No - if given short trials GluA1 mice failed to show a novelty preference with short pre-exposure trials (1-min) Appears as if they do not have a STM - do not have recent spatial memory of arms they just explored Yes - if given 24 hours between pre-exposure and test

Question 45

What does GluA1 dependent rapid synaptic plasticity underlie?

Answer 45

Short-term habituation/ reduction in attention to recently experienced stimuli

Question 46

What do GluA1 independent LTP mechanisms support?

Answer 46

Associative LTM

Question 47

What is the dissociable role of AMPA vs NMDA receptors?

Answer 47

NMDA may be more involved in long-term spatial memory AMPA more involved in short-term habituation