Chapter 6: Reinforcement and Choice

Question 1

Intrinsic reinforcer

Answer 1

obtain reinforcing value while engaging in the behvaiour
intrinsically motivating
social contact
exercise

Question 2

Extrinsic reinforcer

Answer 2

things that are provided as a result of the behaviour to encourage more behaviour in the future

ex. reading in children
the only way to teach kids to read is to get them to read and this usually involved enticing with them with social reinforcement (saying “good job”) or other kinds of external reinforcement

Question 3

More reward does not always mean more

____________. Why?

Answer 3

Reinforcement

bonus for making more parts (i.e. over 50)
only diff between the group that does not get a bonus and the groups that did get a bonus (no difference between the groups who all got bonus but diff bonus values, they all were equally reinforced)

Question 4

aversives can __________ behaviour

Answer 4

reinforce

the aversiveness drives the behaviour!

Question 5

Continuous Reinforcement

Answer 5

Behaviour is reinforced every time it occurs

Question 6

Ratio Schedules

Answer 6

Reinforcer is given after the animal makes the required number of responses

Question 7

Fixed ratio (FR):

Answer 7

fixed ratio between the number of responses made and reinforcers delivered (e.g., FR 10)
• Key elements: postreinforcement pause, ratio run, and ratio strain (going from 1 peck to 100 pecks, subjects tend to stop responding)

see graph slide 9

Question 8

Cumulative Record

Answer 8

Based on old cumulative recorder device (Constant paper output, pen jumps with each response)
rate of responding across time!

Question 9

Variable Ratio (VR):

Answer 9

Different number of responses are required for the delivery of each reinforcer

Value is equal to the average number of
responses made to receive a reinforcer (VR 5)

Responding based on average VR and
minimum

ex. gambling

see graph slide 19 (steep slope of responding
responding at high rate, no postreinforcement pause!)

Question 10

Interval Schedules

Answer 10

Responses are only reinforced if the response

occurs after a certain time interval.

Question 11

Fixed interval (FI):

Answer 11

a response is reinforced only if it occurs more than a set amount of time (responses during the interval don’t matter)

Key elements: fixed interval scallop, limited
hold

i. e. have to wait 10 secs, after 10 secs has elapsed, the first peck at the key will gain them the reward!
ex. cramming before tests

see graph slide 16 (low rates of responding
scallopes responding, post-reinforcement pause!)

Question 12

Variable interval (VI):

Answer 12

responses are reinforced if they occur after a variable interval of time

see graph slide 19

Question 13

Reynolds 1975

Ratio and Interval Schedules Compared

Answer 13

• Compared rates of key pecking of pigeons on
VR and VI schedules
• Opportunities for reinforcement were made
identical for each bird
• The VI bird could receive reward when the VR
bird was within one response of its reward

With equivalent rate of reinforcement, variable ratio schedules produce a higher rate of responding than variable interval schedules

Question 14

Variable schedules produce _________ responding compared to Fixed

Answer 14

Variable schedules produce steadier responding compared to Fixed

fixed = post reinforcement pause

Question 15

Ratio schedules produce ________ of responding than Interval

Answer 15

Ratio schedules produce higher rates of responding than Interval

Question 16

Source of Differences Between Ratio and Interval Schedules:

Differential reinforcement of Inter-response times

Answer 16

Ratio schedules reinforce shorter IRTs

Interval schedules reinforce longer IRTs

Question 17

Source of Differences Between Ratio and Interval Schedules: Feedback function

Answer 17

More feedback (reinforcement) comes with more
responding on Ratio schedules; not so for Interval Schedules (different jobs differ on this aspect)

Question 18

Intermittent Schedules

Answer 18

Fewer reinforcers needed
More resistant to extinction

Variable reinforcement/interval schedules are resistant to intinction

Question 19

Differential reinforcement of high rates (DRH)

Answer 19

Minimum ratio per interval

Question 20

Differential reinforcement of low rates (DRL)

Answer 20

Maximum ratio per interval

to get rid of a behaviour if you don’t hate it but want it to occur less

Question 21

Differential reinforcement of paced rates (DRP)

Answer 21

Maximum ratio per interval (DRH)

Minimum ratio per interval (DRL)

Question 22

Duration Schedules:

Answer 22

response must be made continuously for a period of time

Question 23

Complex schedules:

Answer 23

Conjunctive schedules, Adjusting schedules, Chained schedules…

Question 24

Noncontingent Schedules: Fixed time (FT)

Answer 24

Reinforcer occurs following predictable amount of time regardless of behaviour

Question 25

Choice

Answer 25

Usually considered as a cognitive deliberation - Here measured based on effect of different, concurrent payoff schedules With true and fickle “freedom of choice”, choices would be unpredictable - Understanding choices in terms of consequences allows for prediction with concurrent schedules!

Question 26

Herrnstein, 1961 | matching law

Answer 26

would the animal be able to figure out how to respond based on how much food it gets?? measurement of choice concurrent schedules

Question 27

Measures of choice

Answer 27

1. Relative rate of responding (Behaviour) BL/(BL + BR) BL = rate of responding to left choice BR = rate of responding to right choice (BR + BL) = total responding 2. Relative rate of reinforcement RL/(RL + RR)

Question 28

Matching law

Answer 28

Herrnstein, 1961: Proportion of responding (choice) is equal to the proportion of reinforcement for doing so. There is a correlation between behaviour and the environment. the first equation is = to the second they are proportional = its called the matching law Relative rates of responding match relative rates of reinforcement BL/(BL + BR) = RL/(RL + RR) OR BL/BR = RL/RR

Question 29

Basket ball matching

Answer 29

26 players on a large university basketball team Relative choice of different shot types = relative rate of reinforcement (baskets made) VR, because you may need to make some shots before taking a 3 pt one

Question 30

BL/BR = b(rL/rR)^s | real matching law

Answer 30

real matching law b = bias s = sensitivity Perfect matching, s = 1 • Undermatching, s 1

Question 31

Undermatching

Answer 31

Undermatching, s

Question 32

Overmatching

Answer 32

``` s > 1 – Increased sensitivity to rates of reinforcement – “Stick to the best option” – Common with high cost of switching ``` respond a lot to the best option common if it is costly to switch (long change over delay) will rarely sample from other options that pay off at lower rates

Question 33

Response Bias

Answer 33

Important when there is a difference between operant behaviours - Commonly: side bias Important when there is a choice between reinforcers or responses - Biological predispositions - Quality

Question 34

Matching Law and Simple Schedules

Answer 34

Rate of operant (Bx) and rate other (BO) activities | = Bx/(Bx+Bo) = rx(rx+ro)

Question 35

Matching law describes ________, but does not ________

Answer 35

Matching Law describes the behaviour, but | does not explain it

Question 36

Maximizing theories

Answer 36

Organisms distribute their behaviour so as to obtain the maximum amount of reinforcement over time Explains ratio schedule choice Doesn’t always hold

Question 37

Melioration theories

Answer 37

Making the situation “better” than the recent past Change from one alternative to the next to improve the local rate of reinforcement Animals respond so that the local rate is the same on each alternative Predicted issue: Behaviour is strongly controlled by immediate consequences

Question 38

Serial Reversal Learning

Answer 38

Over many reversals, re-acquisition speeds up Compare with initial acquisition and number of reversals - Behavioural Flexibility

Question 39

Mid-session Reversal

Answer 39

With a reversal part- way through the session: - Perseverative Errors - Anticipatory Errors - Errors shift with changes in time - Pigeons predict reversal based on time - Self-Control?

Question 40

Concurrent Chain Schedules

Answer 40

• Method to determine choice – e.g., whether variety is preferred • Different from concurrent schedules since animals are not free to switch • Able to investigate choice with commitment

Question 41

Self-Control

Answer 41

Commonly used as “willpower” - Circular logic - Describes outcome, not process Better described as: - 1Choice of impulsive vs. delayed options

Question 42

Temporal Self-Control

Answer 42

Choose: Smaller-Sooner reward (SS) vs. Larger-Later reward (LL) • Self-control vs. impulsivity

Question 43

Waiting in Animals

Answer 43

Different species tolerate delays differently e.g., time they will wait for a threefold increase in reward - Temporal Self-Control - chimps are very good at this!

Question 44

Waiting in Humans (Rosati et al. (2007))

Answer 44

- Temporal Self-Control - chimps are very good at this! - chimps are better than humans (expect for when the reward is money)

Question 45

Rachlin & Green (1972)

Answer 45

• Pigeons chose small reward when no delay in Phase 1 • In phase 2, pigeons chose large, delayed reward when T between initial and terminal phases was increased

Question 46

Delay-discounting

Answer 46

Value discounting function : value (V) is directly related to magnitude (M) and inversely related to delay (D), or V = M/(1 + KD) V = value of a reinforcer M = magnitude K = discounting parameter D = delay

Question 47

Madden et al. 1997

Answer 47

•Madden et al. 1997 Opioid-addicted participants steeply discount delayed money and (especially) heroin it is a tendency to give greater value to rewards as they move away from their temporal horizons and towards the "now".

Question 48

Small-But-Cumulative Effects

Answer 48

Malott (1989) Each choice of an SS reward over a LL reward has only a small effect - Builds over time - Difficulty in impulse control - Establishing rules for acceptable vs. unacceptable behaviour - Relapse handling: Dealing with steady stream of temptations

Question 49

Long-term Effects | delayed gratification

Answer 49

``` Mischel: Delayed Gratification Eating the first marshmallow / less- preferred food correlated longitudinally with: - Lower SAT scores - Less educational and professional achievement - Higher rates of drug use ```

Question 50

Simple Self-Control Methods (Skinner)

Answer 50

``` Physical restraint Deprivation/Satiation Distraction DRO Self-Reinforcement Self-Punishment Shaping ```

Question 51

Clinical Implications of self control issues

Answer 51

``` ADHD Predominantly Hyperactive-Impulsive Type Substance abuse disorders Impulsive overeating Other impulse-control disorders Pathological gambling ```