Lecture 9

Question 1

____ ____ is the process whereby organisms learn to make or to refrain from making certain ____ in order to ovtain or avoid certain outcomes

Answer 1

Operant conditioning

Question 2

Who is Edward Throndike?

Answer 2

A psychologist who, at the end of the 19th century, first tried to systematically explore how animals learn new behaviors with his puzzle boxes

Question 3

Operant conditioning:
In the presence of a particular stimulus, called the ____ ____ (____), a particular ____ (____) may lead to a particular ____ (____)

Answer 3

discriminative stimulus (SD), response (R), outcome (O)

Question 4

Operant conditioning can be formulated as a three-part association: ____ ____ (____) -> ____ (____) –> ____ (____)

Answer 4

Discriminative stimulus (SD), Response (R), Outcome (O)

Question 5

What are the SD, R, and O in the case of a puzzle box?

Answer 5

SD = the box
R = sequence of movements that open the door
O = escaping from the box

Question 6

Fill in the blanks using SD, R and O (some may be used more than once)
The association between ____ and ____ is strengthened when ____ is followed by a desirable ____

Answer 6

SD; R; R; Outcome

Question 7

____-____ ____ is an operant conditioning paradigm in which the experimenter defines the beginning and end points of each trial

Answer 7

Discrete-trials paradigm

Question 8

____-____ ____ is an operant conditioning paradigm in which the animal can operate the apparatus as it chooses in order to obtain reinforcement (or avoid punishment)

Answer 8

Free-operant paradigm

Question 9

B.F.Skinner devised a cage in order to utilize ____-____ ____, which typically consist of one or more ____, food port, one or more ____, and can be further equipped with metal grid floor to give a ____ and ____ speaker.

Answer 9

free-operant paradigm; levers; lights; shock; audio

Question 10

What are the SD, R, and O in the case of a Skinner chamber?

Answer 10

SD = the chamber
R = pressing the lever
O = receiving food

Question 11

What is the dependent variable in the case of a Skinner chamber?

Answer 11

number of lever presses (as a rate of responses or cumulative responses)

Question 12

____ ____ help the learner identify (or “discriminate”) the conditions in which a response will be followed by a particular outcome

Answer 12

Discriminative stimuli

Question 13

In operant conditioning, ____ is defined not by a particular ____ of motor actions but rather by the ____ it produces

Answer 13

response; pattern; outcome

Question 14

____ is an operant conditioning technique in which successive approximations to the desired response are reinforced
(criteria for response becomes increasingly specific as trials go on)

Answer 14

Shaping

Question 15

____ is an operant conditioning technique in which organisms are graually trained to execute complicated sequences of discrete responses

Answer 15

Chaining

Question 16

____ is an outcome of behavior that leads to the increased likelihood of that behavior in the future

Answer 16

Reinforcer

Question 17

____ is an outcome of behavior that leads to the decreased likelihood of that behavior in the future

Answer 17

Punisher

Question 18

____ is the process of providing outcomes (____) that lead to increased probability of a behavior

Answer 18

Reinforcement; reinforcers

Question 19

____ is the process of providing outcomes (____) that lead to decreased probability of a behavior

Answer 19

Punishment; punishers

Question 20

____ ____ are rewards that directly affect the organism’s physical well-being, including food, water, sleep, comfortable temperatues, and sex

Answer 20

primary reinforcers

Question 21

Calrk Hull’s ____ ____ ____ proposed that all learning reflects the innate, biological need to obtain ____ ____

Answer 21

drive reduction theory; primary reinforcers

Question 22

The ____ ____ ____ is when a normally acceptable reinforcer evokes less responding if a preferred reinforcer is expected

Answer 22

negative contrast effect
（aka我本可以忍受黑暗，如果我不曾见过太阳）
(ex. if primed with sweetened water, less sucking plain water will be observed in later trials)

Question 23

____ ____ are reinforcers that initially has no biological value but that has been paired with a primary reinforcer. By virture of being paired with a primary reinforcer, these become reinforcers themselves, which the organism will work to obtain

Answer 23

Secondary reinforcers
(the conditioned stimulus)
(like in dogs: clicker = food)
(like in humans: status = health/wealth/sex)

Question 24

Just as organisms will work to obtain ____, they will also work to avoid ____. However, both Throndike and Skinner conclude that ____ is not nearly as effective as ____ at controlling behavior

Answer 24

reinforcers; punishers; punishers; reinforcers

Question 25

There are several factors that determines how effective the punishment will be: ____ leads to more ____ behavior, ____ ____ for punishment can encourage cheating, concurrent ____ can undermine it, and initial intensity matters

Answer 25

punishment; variable; Discriminative stimuli; reinforcement

Question 26

Differential reinforcement of alternative behaviors (BRA)

Answer 26

Instead of delivering punishment each time an unwanted behavior is exhibited, reward preferred alternative behaviors (ex. children with autism/developmental disorders show persistent habits of self-injurious behaivor, and rather than punishing the child for unwanted behavior, they are rewarded everytime they do a desired behavior)

Question 27

Normally, ____ outcomes produce the fastest learning. But if there is a long ____ between the response and outcome, it is more likely that other behaviors have taken place during the interval, and those behaviors might thus be ____ with the outcome.

Answer 27

immediate; delay; associated

Question 28

Rats trained to lever-press with food reinforcement learned more quickly when there is ____ delay and ____-second delay, but learned slowly with a ____-second delay

Answer 28

no; 4; 10

Question 29

____ ____ is when the response causes a desirable element to be "added" to the environment, which causes the response to be more frequent

Answer 29

positive reinforcement

Question 30

____ ____ is when the response causes an undesirable element to be "added" to the environment, which causes the response to be less frequent

Answer 30

Positive punishment

Question 31

____ ____ is when the response causes an undesirable element to be "subtracted" from the environment, making the response to be more frequent

Answer 31

negative reinforcement

Question 32

____ ____ is when the response causes a desirable element to be "subtracted" from the environment, making the response to be less frequent

Answer 32

negative punishment

Question 33

____ ____ is the rules determining how and when outcomes are delivered in an experiment

Answer 33

Reinforcement schedules

Question 34

____ reinforcement schedule follows every instance of the response by the reinforcer, whereas ____ reinforcement schedule follow only some instances of the response by the reinforcer

Answer 34

Continuous; partial

Question 35

____-____ (FR) schedule requires a specific number of ____ must occur before a ____ is delivered

Answer 35

Fixed-ratio; responses; reinforcer

Question 36

What does Fixed-ratio 5 (FR5) mean?

Answer 36

Reinforcement arrives after every fifth response

Question 37

____-____ (FI) schedule requires the first ____ is made after a fixed amount of ____ to be reinforced

Answer 37

Fixed-interval; response; time

Question 38

What does Fixed Interval 1-min (FI1) mean?

Answer 38

Reinforcement follows the first response made after a 1-minute interval since the last reinforcement

Question 39

____-____ (VR) schedule requires that a specific number of ____, on average, must occur before a ____ is delivered

Answer 39

Variable-ratio; response; reinforcer

Question 40

What does Variable Ratio 5 (VR5) mean?

Answer 40

On average, every fifth response is reinforced (doesn't have to be exactly 5)

Question 41

____-____(VI) schedule requires the first ____ on average after a variable amount of ____ to be reinforced

Answer 41

Variable-interval; response; time

Question 42

What does variable interval 1-min (VI1) mean?

Answer 42

Reinforcement follows the first response made after a 1 minute interval, on average, since the last reinforcement

Question 43

____-____ 5 schedule shows the fastest increse in number of responses, and the number of responses of____-____ 5 schedule is flatter and plateaus in between

Answer 43

Variable-ratio; fixed-ratio

Question 44

____-____ 10 sec schedules have a similar slope with ____-____ 10 sec schedules, except that ____-____ schedule shows continuous increase in cumulative response, whereas in ____-____ schedule the cumulative response increases at the interval and slows down in between

Answer 44

variable-interval; fixed-interval; variable-interval; fixed-interval

Question 45

*Use abbreviated form* ____ and ____ schedules tend to produce higher rates of responding than ____ and ____ schedules.

Answer 45

VR; VI; FR; FI

Question 46

____ ____ schdules is one that the organism can choose between several possible responses, each leading to a different outcome. This schedule is used to study ____ behavior

Answer 46

concurrent reinforcement; choice (like 剧情向游戏)

Question 47

In choice behavior/concurrent reinforcement schedules, an organism's response patterns will closely mimic the relative rates of reinforcement for each possible response. This phenomenon is called...

Answer 47

Matching law of choice behavior

Question 49

____ ____ is the study of how organisms allcate their time and resources among possible options

Answer 49

Behavioral economics

Question 50

____ ____ is the particular allocation of resources that provices maximal subjective value to an individual

Answer 50

Bliss point

Question 51

____ ____ is the progressive reduction of the *subjective value* of a reward the longer it is delayed. ex. It is easy to convince a student to put off fun in order to study if the exam is tomottow, but harder if the exam is not for 5 weeks since the event is in further future

Answer 51

Delay discounting

Question 52

There are two levers to choose from: one that delivers a small immediate reward, and one that delivers a larger reward after some delay. How would a rat behave in either short delay and long delay situations?

Answer 52

Short delay (0-5sec): rats would choose the larger reward on nearly 100% of choice trials Longer delay (20-30sec): Increasingly less likely to wait, opting more often for the smaller, immediate reward

Question 53

Skinner advocated an extreme form of behaviorism called ____ ____, in which he asserted hat consciousness and free will are illusions, and argued that humans, like all other animals, function by blindly producing learned responses to environmental stimuli.

Answer 53

Radical behaviorism