Instrumental conditioning

Question 1

Miller and Konorski (1928) during CS-US training added a second unrelated action

Answer 1

Animals learnt to make the action even though it was unrelated to the US.

Question 2

Reinforcement in instrumental (operant) conditioning

Answer 2

Increases behaviour. Positive (add appetitive stimulus) or negative (escape by removing noxious stimulus after correct behaviour/active avoidance)

Question 3

Punishment in operant conditioning

Answer 3

Decrease behaviour. Positive (add noxious stimuli) or negative (remove appetitive stimulus following behaviour).

Question 4

Shaping by successive approximations

Answer 4

Complex behaviours can be built up through small steps

Question 5

Ratio vs. interval reinforcement

Answer 5

1. Fixed ratio: highest rate of responding. Reinforce after a fixed number of responses.
2. Variable ratio: high rate. Reinforce after an unpredictable number of responses.
3. Fixed interval: scalloped shape. Rapid responding near time for reinforcement. Reinforce after a fixed amount of time.
4. Variable interval: steady, moderate responding.

Question 6

Dixon et al. (2014) slot machines

Answer 6

Micro-reinforcement can speed up actions. 20-line slot machines (micro-rewards with a false sense of progress; variable ratio reward) reduced the post-reinforcement pause compared to 1-line slot machine. High-risk gamblers reported a greater sense of control.

Question 7

Goal-directed vs. habitual behaviour

Answer 7

Goal-directed: actions performed because of their expected outcomes. (Not just drives, but outcome).
Habits: actions triggered by stimuli regardless of the outcome’s current value. (Response-outcome associations).

Question 8

Hershberger’s (1986) Pavlovian chicken

Answer 8

Chick learnt the visual approach signal for food. It fails to learn to move backwards (failure of goal-directed behaviour).

Question 9

Dickinson and Balleine’s (1994) 2 criteria for goal-directedness

Answer 9

1. Instrumental criterion. Animal must learn contingency between response and outcome.
2. Goal criterion. Animal must learn incentive value of a stimulus.

Question 10

Instrumental criterion

Answer 10

Animal learns that pressing a lever gives it food.

Question 11

Goal criterion

Answer 11

A representation of the outcome is a goal for the agent. Animal must learn the incentive value of a stimulus (eg. food reduces hunger). If the animal stops pressing the lever when it’s full, this suggests it values the outcome appropriately.

Question 12

Colwill and rescorla (1985) reinforcer devaluation and instrumental responding

Answer 12

Rats reduced response when outcome was devalued (made aversive/fed to satiation). Response shows sensitivity to changes in outcome. (Absence of devaluation effect suggests that behaviour controlled by habit and not R-O).

Question 13

Valentin et al. (2007) human goal-directed learning

Answer 13

OFC shows modulatory activity during selection of devalued (less activity)/non-devalued (more activity) outcomes. Participants avoided devalued outcome cue.

Question 14

2 candidate theories for ABA effect

Answer 14

1. Direct effect: exercise reduces reinforcing value of food/food not as salient or pleasurable etc.
2. Indirect effect: behavioural failure to adapt to restricted feeding schedule (more likely indirect effects).

Question 15

Which animals are more vulnerable to ABA effect?

Answer 15

Animals that develop food anticipatory behaviours (more movement before food).

Question 16

Robinette et al. (2021) serotonin SIRT1 and ABA

Answer 16

Serotonin may delay the onset and progression of ABA in animals. SIRT1 KO mice lost less weight. SIRT1 overexpressors lost more weight.

Question 17

What does SIRT1 do?

Answer 17

SIRT1 increases foraging and activity (when hungry) and decreases serotonin.

Question 18

Dwyer and Boakes (1997) 3 experiments ABA recovery of weight

Answer 18

1. Access to feeding schedule prior to introduction of access to the wheel.
2. Move feeding to start of rats’ active period.
3. Activity restricted before feeding.

Supports indirect theory as food hasn’t become less rewarding, rats just failed to adapt to feeding schedule.

Question 19

Balleine et al. (2009) motivation and incentive learning

Answer 19

Not just need learning R->O but also that outcome changes motivational state (food reduces hunger). Rats don’t just respond based on current motivation (hunger), they also need prior experience of the outcome’s value in that motivational state. Rats trained while full don’t know that food reduces hunger, so they don’t press the lever when they later get hungery.

Question 20

Brain regions for goal-directed action-outcome learning vs. habitual performance

Answer 20

Goal-directed: dorsomedial striatum.
Habitual: dorsolateral striatum.