final

Question 1

conditioned inhibition

Answer 1

the CS announces no US

Question 2

conditioned inhibition example

Answer 2

good response > click!; wrong response > no, thank you!

Question 3

conditioned inhibition dog trainers

Answer 3

call this a “non-reward marker” — something signals nothing will happen (good or bad)

Question 4

safety signals

Answer 4

can be used to calm down — when this is present, nothing happens

Question 5

conditioned inhibition definition

Answer 5

procedure in which a CR elicited by a CS is inhibited when another concurrently trained CS signals the absence of a US

Question 6

CS can be…

Answer 6

xcitors or inhibitors — these constructs are crucial in our discussion of motivation (can either slow things down or accelerate things)

Question 7

conditioned excitation

Answer 7

CS being associated with a US

Question 8

Method 1 (intuitive): differential inhibition or discriminative inhibition procedure

Answer 8

classical conditioning, simply pairing with a US or not (same with habituation/dishabituation). Based on a pairing

Question 9

differential/discriminative inhibition pairings

Answer 9

CSa <> US
CSx <> no US
CSa = clicker, CSx = note

Question 10

differential/discriminative inhibition animal responds to

Answer 10

Animal responds only to CSa…in other words, it can discriminate between the two

Question 11

differential/discriminative inhibition: two sounds

Answer 11

Trying to discriminate between two sounds. Condition one with food and the other without. Eg. If the animal is salivating, C, therefore not G:
What if CSa is C (musical note “do”) and CSx is G (musical note “sol”)

Question 12

Method 2 (could be competing): conditioned inhibition procedure

Answer 12

getting two reinforced stimuli to compete

Question 13

conditioned inhibition procedure trials

Answer 13

US is presented (reinforced trials): CSa > US
US is omitted (non reinforced trials) CSa + CSx > nothing

Question 14

conditioned inhibition outcome

Answer 14

Subjects respond to CSa
Subjects do not respond to CSa + CSx
Subjects also do not respond to CSx (= conditioned inhibitor!)
CSx by itself signals no US (food)

Question 15

conditioned inhibition CSx

Answer 15

CSx on its own was never on its own predicting the food.

Question 16

conditioned inhibition designed by

Answer 16

Pavlov

Question 17

information value in conditioning

Answer 17

CS provides information on the US

Question 18

rescorla experiments

Answer 18

contingencies between CS (tones) and US (shock)

Question 19

rescorla experiments groups

Answer 19

3 groups:
positive, zero, negative contingency

Question 20

what is key in rescorla

Answer 20

probability of CS followed by US

Question 21

rescorla positive contingency

Answer 21

if CS is followed by US in a relatively predictable way — excitation

Question 22

rescorla zero contingency

Answer 22

completely unpredictable

Question 23

rescorla negative contingency

Answer 23

inhibition, reverse of number one. If you have a US that is not immediately preceded by the CS (almost like backward conditioning) — inhibition

Question 24

appetitive conditioning

Answer 24

cues can announce something pleasurable

Question 25

appetitive conditioning example

Answer 25

Coyotes associate the human scent or voice with food (litter on trails)

Question 26

aversive conditioning

Answer 26

Cues can announce something not pleasurable

Question 27

aversive conditioning example

Answer 27

Humans use human scent with hazing methods to scare the coyotes away

Question 28

powerful appetitive USs

Answer 28

food, sex, drugs

Question 29

aversive control, appetitive conditioning

Answer 29

get coyotes afraid of what they are supposed to avoid. Coyotes predicted the presence of food with humans, food and humans became the same thing — this classical conditioning was the problem. Became operant when the coyotes actually acted on the human to obtain to the food (an attack)

Question 30

aversive conditioning, hazing program

Answer 30

hazing program may not be successful with coyotes that already formed that association. Could be useful in young coyotes, to begin to conditioning coyotes that humans are dangerous. Have to be consistent in these pairings.

Question 31

main method of extinction

Answer 31

you stop pairing of CS-US

Question 32

is extinction unlearning or forgetting?

Answer 32

yes, but also learning a new response — learning what happens when CS is followed by US of one nature and of another nature

Question 33

extinction is actually

Answer 33

Responses are suppressed or not expressed

Question 34

extinction is likely just..

Answer 34

a suppression of response. It is not permanent, if the right conditions are back together the response comes back

Question 35

renewal effect

Answer 35

CS-US pairing in context X Extinction in context Y — now bringing the organism to a different environment on purpose to extinguish the response CS in context X > renewed

Question 36

renewal effect two associations

Answer 36

1. CS — US 2. CS — no US

Question 37

extinguishing is...

Answer 37

is context specific — extinction must be done in context of acquisition

Question 38

extinguishing is important in

Answer 38

therapy: eg. Might be able to handle it in the therapists office but not in the real environment If you experience of fear of explosions in war, being exposed just to the sound in a clinic may not help you

Question 39

extinction is NOT

Answer 39

loss of information

Question 40

compound conditioning/CS

Answer 40

The presentation of 2 or more CSs, ether - simultaneous - serial

Question 41

simultaenous compound CS

Answer 41

CSs presented at same time

Question 42

serial compound CS

Answer 42

CSs presented in a sequence

Question 43

compound conditioning example

Answer 43

Tone + light > US (food) This will condition normally, as if one S But, if later, the tone is tested alone…the CR won’t be as strong — compound stimuli will be stronger If you really want a good response to the tone, condition the tone alone

Question 44

isolating one part of compound conditioning suggests that

Answer 44

cues CSs are competing

Question 45

compound conditoning hints at

Answer 45

the importance of having salient cues

Question 46

salient cues

Answer 46

Salient cues = more intense. The more intense cue is more likely to get conditioned and be noticed

Question 47

importance of saliency related to what theory

Answer 47

mackintosh attentional theory of conditioning

Question 48

CS competition — how to win the competition?

Answer 48

1. Be more noticeable > overshadowing (because of saliency) 2. Be there sooner > blocking (because of temporal priority) — when something happens first, in the sequence of conditioning

Question 49

overshadowing

Answer 49

- More salient cues get more learning in compound conditioning, still less than if trained alone - This brings us back to the idea of an information value of cues (cognitive interpretation): we use here the concept of information - If you really want to have a clear effect of one CS, try to isolate that completely and train it alone Neural networks/information theory involved here in classical conditioning

Question 50

compound conditoning example

Answer 50

CSa + CSb > US is compound conditioning, ie. competing CSs

Question 51

when two CSs compete, if one is more salient it will...

Answer 51

overshadow the less salient ones

Question 52

compound conditioning analogy

Answer 52

Picture book for kids. The book contains text and pictures, but the pictures will most likely overshadow the text In other words, learning to read this way is not optimal

Question 53

compound conditioning spatial learning

Answer 53

Similarly, in spatial navigation, landmark cues may overshadow more general spatial cues — most of the time when you navigate proximal cues influence you most. The most important cues are the most salient

Question 54

compound potentiation

Answer 54

This is the opposite of overshadowing In this case, there is more conditioning to a weak conditioned stimulus if it is combined with a more salient conditioned stimulus during conditioning

Question 55

compound potentiation example

Answer 55

The conditioning of a weak odour may be strong if it is combined with a salient taste during conditioning Eg. Gadbois aversion for even the smell of whisky in his taste aversion for whisky Taste and odour go together — they are intimately linked stimuli. Smell is usually the early CS that something good/bad is coming.

Question 56

kamins blocking effect example scenario

Answer 56

Scenario: visit in-laws every Sunday. Your FIL offers an anise based liqueur, but you drink it every time even though it makes you sick. One day, he offers you bagels with the anise. Will you develop an aversion to bagels? No. Even if you get a bit sick, you will attribute your malaise to the anise not the bagels.

Question 57

kamins blocking effect assigned variables

Answer 57

CSa: anise Cab: bagels

Question 58

kamin blocking

Answer 58

you were exposed to the aversive first, then you combined the bagel (neutral). Only the aversive maintains the negative response.

Question 59

kamin original experiment

Answer 59

The experimental group gets the pairing as described above When bagels are presented without anise (Test phase) almost no CR Less response (CR) in the experimental group than the control group

Question 60

kamin what does prior conditoning do

Answer 60

prior conditioning with the anise has blocked conditioning to the bagels Basically, the anise already predicted the sickness. Adding the bagels did nothing.

Question 61

kamin what is the control group

Answer 61

the basic compound conditioning explained previously

Question 62

original experiment control group

Answer 62

compound conditioning group

Question 63

original experiment experimental group

Answer 63

pre-trained

Question 64

table in slides

Answer 64

basics of classical conditioning slide 56

Question 65

Is overshadowing different from blocking?

Answer 65

It occurs in only one conditioning phase

Question 66

overshadowing application

Answer 66

See application mentioned by Boynton for chemotherapy: scapegoating Give patients a salient new flavour just before chemo (eg. Ginger candy) This new flavour will overshadow any other flavour eaten in recent meals (just before the chemo) The ginger candy becomes the scapegoat…problem is…the patient may never be able to stomach ginger candies in the future

Question 67

information provider

Answer 67

- The CS-US association is an information provider unless the informational value is blocked - to be useful, a cue needs to be non-redundant (eg. Needs to be a non-redundant predictor) - Redundant cues are typically not very informant

Question 68

learning only occurs if...

Answer 68

the CS provides new information on the US

Question 69

CSs may

Answer 69

compete with each other

Question 70

a more informative CS will win over

Answer 70

a less informative CS

Question 71

Two areas of experimental psych have produced solid quantitative models, even “laws”

Answer 71

- Psychophysics and the study of sensation and perception - Learning theory and the study of conditioning and learning

Question 72

The Rescorla-Wagner model

Answer 72

- most well known mathematical model in experimental psych and has been applied beyond learning theory - Tries to explain blocking - A lot about the unconditioned stimulus

Question 73

The Mackintosh model

Answer 73

- A lot about conditioned stimulus - Attentional theory (saliency)

Question 74

The Pearce-Hall model (including Kaye & Pearce)

Answer 74

US and CS modulation, attention, negative transfer

Question 75

Wagner's original model (SOP) and AESOP

Answer 75

Priming (Attentional priming), affective dimension

Question 76

Rescorla-Wagner model

Answer 76

We know that cues compete with each other for associative strength We know and will reaffirm that contiguity (in time and space) is not the full story. Information produced by the cues needs to be: Reliable Non-redundant

Question 77

Rescorla-Wagner model counterintuitive

Answer 77

difference between phase 1 and 2. In all experiments in learning theory we have an acquisition phase and after. In acquisition phase, you do want that CS/US pairing to be tight and predictable. Later, if the CS always announces the US, your animal loses attention (redundancy). Then work with the saliency of the CS/US or make it less predictable. Uncertainty is what facilitates learning after a while (NOT initially)

Question 78

error detection and error correction

Answer 78

1. Prediction error: The difference between what is predicted, and what actually happens 2. Error detection: Ability to detect errors or discrepancies between the predicted and actual occurrences (detect prediction errors) 3. Error correction learning: Similar to “trial and learning”. Learning to reduce the error or discrepancy between predicted and actual occurrences

Question 79

error correction learning

Answer 79

- This type of learning applies to classical conditioning and many other types of learning, ie. skill and motor learning - This includes prediction error and error detection

Question 80

pavlovian classical experiment follows this scheme:

Answer 80

Bell — food, Salivate salivate Or CS — US, CR UR

Question 81

when conditioning to one CS generalizes to another we have...

Answer 81

generalization

Question 82

generalization graphic

Answer 82

Question 83

Historically: two views on classical conditioning

Answer 83

SR learning SS learning

Question 84

SR learning

Answer 84

CS-UR ~ note that this form of learning is known to occur in some case

Question 85

SS learning

Answer 85

US » CS = stimulus substitution theory or "cognitive" theories of classical conditioning; this view was confirmed by Rescorla in 1978.

Question 86

SR vs. SS learning graphic

Answer 86

Question 87

Is it possible to respond to a CS (e.g., tone) that was never paired with the US (e.g., food)? (method)

Answer 87

One method can help answer this question: US devaluation

Question 88

Is it possible to respond to a CS (e.g., tone) that was never paired with the US (e.g., food)? (forms of learning)

Answer 88

- Second order or higher order conditioning - Sensory pre-conditioning

Question 89

conclusion about US-CS?

Answer 89

association is essential!

Question 90

figure from textbook?

Answer 90

???

Question 91

is sign tracking resistant to outcome devaluation?

Answer 91

Sign tracking but not goal tracking is resistant to outcome devaluation

Question 92

We will see later that cues paired with a US/reward can...

Answer 92

gain "incentive salience"

Question 93

what happens if you get a temporary seperation of the CS and US in classical condtioning

Answer 93

you will decrease sign tracking, but increase goal tracking.

Question 94

second-order condiotning and sensory preconditoning graphics

Answer 94

Question 95

conclusion, learning can occur...

Answer 95

without an US > pro-SR learning

Question 96

second order conditioning example

Answer 96

You get bitten (US) by a dog (CS) » Fear (CR) The later you associate the park (CS2) with dogs and get fearful (CR) entering the park

Question 97

second order conditoning graphic

Answer 97

Question 98

sensory preconditioning graphic

Answer 98

Question 99

sensory preconditioning appears to be...

Answer 99

pro SR learning

Question 100

sensory preconditioning analogy

Answer 100

Guilt by association" analogy - Peter and John hang-out together. Peter is arrested for using cocaine. You assume John is using cocaine.

Question 101

what happens with sensory preconditioning

Answer 101

wo neutral S get associated: The target CS (tone) never gets paired with the response it evokes. In sensory preconditioning, the same two stimuli are paired before the light can even elicit a CR. So with that logic, it is a case of S-S learning.

Question 102

sensory preconditioning is therefore

Answer 102

pure S-S learning

Question 103

generalization

Answer 103

concise def ???

Question 104

higher (second) order conditioning

Answer 104

concise def ???

Question 105

sensory preconditoning

Answer 105

concise def ???

Question 106

methods in classical conditioning: motor responses

Answer 106

- Proboscis Extension Response (PER) conditioning to artificial odours in bees - Eyeblink conditioning in rabbits

Question 107

methods in classical conditioning: emotional responses

Answer 107

- Fear conditioning in rats - Autoshaping in pigeons

Question 108

methods in classical conditioning: motivational responses

Answer 108

- Appetitive conditioning in rats - Taste aversion learning

Question 109

eyeblink conditioning in rabbits associations

Answer 109

UR = eyeblink CR = eyeblink US = puff of air CS = tone/light

Question 110

fear conditioning/threat conditioning/conditioned emotional response/conditioned suppression associations

Answer 110

UR = bar pressing (from training) CR = fear/suppression of bar pressing US = shock CS = tone/light/noise

Question 111

conditioned fear

Answer 111

the fear response is often "freezing".

Question 112

the CS suppresses the bar pressing, therefore

Answer 112

The response (pressing the bar) is suppressed.

Question 113

in conditioned fear, the suppresion is

Answer 113

the dependent variable, i.e., a measure of conditioned fear » suppression ratio

Question 114

suppression ratio

Answer 114

responding CS/(responding CS + responding pre-CS)

Question 115

When the CS does not change the bar pressing rate, the S.R

Answer 115

= 0.5. A complete suppression is 0.

Question 116

fear conditioning is an important concept in...

Answer 116

CBT for fear and anxiety disorders, even some elements of PTSD.

Question 117

conditioned emotions are important in...

Answer 117

motivational aspects of instrumental conditioning

Question 118

autoshaping in pigeons: sign tracking

Answer 118

focus on the CS's, or cues predicting the US.

Question 119

autoshaping in pigeons: goal tacking

Answer 119

focus on the USs

Question 120

autoshaping therefore is actually

Answer 120

sign tracking

Question 121

autoshaping is a convenient way to train animals in

Answer 121

Skinner boxes. Alternative: successive approximations

Question 122

signal-food conditioning protocol

Answer 122

- You light the "pecking key" for a few seconds, then the pellet magazine opens. - After a number of pairings (45 or so, more or less), the pigeons will peck at the key (it announces the delivery of food, and it is signtracking... that cue becomes associated with the food). Note: the food delivery is not contingent upon the pecking. The pigeons does not have to peck the key.

Question 123

appetitive conditoning in rats

Answer 123

- This is the "magazine approach procedure": CS (any) - US (food) - The approach is more likely to happen when the CS is produced. This is goal tracking as defined earlier.

Question 124

taste aversion learning associations

Answer 124

UR = nausea CR = nausea US = drug injection (emetic drug or nausea/vomiting inducing) CS = flavour S (saccharin)

Question 125

taste aversion learning CS/US

Answer 125

CS-US can be separated in time by hours and still the conditioning will take place.

Question 126

taste aversion clear and dramatic example

Answer 126

Clear and dramatic example in humans: Observed in humans with chemotherapy as well.

Question 127

strength of conditioning factors

Answer 127

Time / timing Novelty of CS and US Intensity of CS and US Pseudoconditioning

Question 128

four main types of CS/US associations

Answer 128

1. Delay conditioning - Short-delay - Long-delay 2. Trace conditioning 3. Simultaneous conditioning 4. Backward conditioning - Other factor: Trial spacing

Question 129

types of conditioning graphic

Answer 129

Question 130

conditoning works better if...

Answer 130

the CS occurs before the US .

Question 131

the CS _____ the US

Answer 131

announces

Question 132

does backward condtioning work well?

Answer 132

typically no

Question 133

delay conditoning

Answer 133

The interval of 1me depends on the type of conditioning.

Question 134

delay conditoning example

Answer 134

Example: Eyeblink (very short) vs taste aversion (very long) conditioning.

Question 135

trace conditoning

Answer 135

Gap between CS and US = "trace interval"

Question 136

is trace conditioning good

Answer 136

Fine procedure, but decreased efficiency with increased delay.

Question 137

trace conditoning possible issues

Answer 137

- The memory of the CS. - Inability for the animal to differentiate between the trace interval from the time between trials.

Question 138

simultaneous condtioning

Answer 138

Full temporal overlap of the CS and US.

Question 139

simultaneous condtioning validity

Answer 139

Usually not great results, but some exceptions.

Question 140

backward conditoning

Answer 140

May be a signal of "no US" (i.e., a conditioned inhibitor).

Question 141

backward conditioning may be associated with...

Answer 141

with relief in fear conditioning. = “safety signal”

Question 142

conditioning works best if...

Answer 142

trials are spaced-out over time (spaced trials); The ITI is crucial.

Question 143

ISI?

Answer 143

interstimulus interval

Question 144

ITI

Answer 144

intertrial interval

Question 145

are massed trials efective

Answer 145

no

Question 146

summary of conditioning

Answer 146

Short ISI’s Long ITI’s

Question 147

ratio between ITI and duration of CS graphic

Answer 147

Question 148

novelty of the CS and US is...

Answer 148

very important: Pre-exposure to CS and US before conditioning can interfere with learning.

Question 149

CS pre-exposure

Answer 149

"latent inhibition". You can habituate to a potential CS with repeated exposures.

Question 150

US pre-exposure

Answer 150

Randich & LoLordo . Will delay subsequent conditioning. You can habituate to a potential US with repeated exposures.

Question 151

CS and US pre-exposure both have

Answer 151

real-life implications, e.g., for fear/phobia acquisition.

Question 152

intensity of the CS/US: US

Answer 152

The CR will be strong, if the US is strong. So the intensity of food or shocks, etc., will influence the magnitude of the response.

Question 153

intensity of the CS/US: CS

Answer 153

Strong salient CS's will positively influence learning as well. So a strong tone, or flash of light, etc., will influence the magnitude of the response.

Question 154

what is best for intensity of CS/US?

Answer 154

With CS's salient, but not scary (overpowering) is good.

Question 155

Counterfeit (fake) conditioning: Pseudoconditioning and sensitization : What if the CS elicits a response like the one that you are trying to condition? (Example)

Answer 155

You use CS = light. Response (UR) = blinking. Natural blinking or conditioned blinking? So what is going on? » impossible to tell

Question 156

Counterfeit (fake) conditioning: Pseudoconditioning and sensitization : culprit 1

Answer 156

Sensitization

Question 157

Counterfeit (fake) conditioning: Pseudoconditioning and sensitization : culprit 2

Answer 157

Pseudoconditioning

Question 158

counterfeit/pseudo: sensitization example

Answer 158

Blinking for the camera. Initially a natural response, can get sensitized and anticipated.

Question 159

counterfeit/pseudo: psuedoconditoning example

Answer 159

Mere exposure to the US! "Increased responding that may occur to a stimulus whose presentations are intermixed with presentation of a US in the absence of the establishment of an association between the stimulus and the US". (Domjan, 2015) Experiments need to control for both sensitization and pseudoconditioning.

Question 160

Rescorla and Wagner model: If no CS or a novel CS is followed by a US

Answer 160

- The US will be unexpected - You have a positive prediction error - The larger the error, the greater the learning!

Question 161

Rescorla and Wagner model: If a well learned CS is followed by the expected US

Answer 161

- The US is expected… - There is no error prediction (because no discrepancy) - There is no learning.

Question 162

Rescorla and Wagner model: If there is a CS-US association established, but then no US occurs

Answer 162

- The absence of the US will be unexpected - You have a negative prediction error - There is a decrease in CS-US association - This is extinction

Question 163

What happens if you trained a dog every time the CS is produced the US follows (clicker training, phase 1)

Answer 163

he theory is that technically it would suggest the CS (clicker) itself is reinforcing. After a while the dog might not even need the treat, the click is reinforced enough. If the association between the US/CS is weakened, therefore the clicker would not become as good of a predictor overtime and would lose its reinforcing value.

Question 164

Rescorla and Wagner ___ predict that the CS can be the ____ itself

Answer 164

do not, reward

Question 165

rescorla and wagner...

Answer 165

do not think clicker training would work if the clicker is used in isolation

Question 166

Basic assumptions of RS

Answer 166

- Each CS has an association weight, or strength of association, with the US. - Cues (CS’s) compete for associative strength. - This tells us that information, or the surprise value of the stimuli involved in important

Question 167

key element in RW

Answer 167

the element of suprise!

Question 168

what is surprise RW?

Answer 168

- What is surprise? It is what is unexpected. - Part of this may be about attention — how much attention are you giving to elements in that model - Either you think its coming or its not, or vice versa, or a completely novel experience

Question 169

premise of surprise RW

Answer 169

The US needs to be "surprising". With time, the US, in any conditioning sessions, becomes less and less "surprising".

Question 170

why is surprise needed?

Answer 170

Why? Because the CS becomes a solid predictor of the US.

Question 171

example of CS being solid predictor o US

Answer 171

clicker training in dogs

Question 172

learning wont occur anymore when...

Answer 172

the CS predicts the US perfectly.

Question 173

suprise clicker training example

Answer 173

Clicker training, your click pouch should be full of whole bunch of different intensity treats (surprise element). Slight variation of the US is good — this is for appetitive learning Could apply to a person as well. If a CS announces a shock, same principles apply especially if it is always the same small shock. If you start varying the intensity of the shock, then the person will start paying attention again.

Question 174

If the CS/US link is highly predicable there is...

Answer 174

no learning, learning depends on not being exactly sure how that works.

Question 175

what happens with repetition in learning

Answer 175

Repetitions get boring after a while, if you can always predict things then you start getting bored. That means though that you have already learned the task. But how do you keep a strong response/motivation — this is where unpredictability might help - You don’t want the response to start to extinguish - But if you maintain the link between CS/US it shouldn’t extinguish

Question 176

how do you avoid extinguishing of connection

Answer 176

maintain the link between CS/US

Question 177

Does it matter if “learning does not occur anymore”?

Answer 177

you are maintaining the strength of the association… so maybe not.

Question 178

Does that mean that you can drop the US?

Answer 178

- Then you create a negative prediction error (see above) - There is a decrease in CS-US association… - So a US is important (at least some of the time…)

Question 179

what is important in surprise/learning

Answer 179

Uncertainty/intermittent reward is good for this

Question 180

If you drop the US you get...

Answer 180

a negative prediction error, you are affecting the CS/US relationship

Question 181

mathematical model, a.k.a

Answer 181

the neural network-like model:

Question 182

mathematicla model peak of learning is

Answer 182

the associative strength between the US and the CS.

Question 183

how is associative strength developed

Answer 183

Based on strengthening of the relationship between the CS and US. The more you do click/treat, with time the dog learns that the click predicts the treat.

Question 184

learning ceiling is denoted by ____, it is

Answer 184

This learning ceiling effect is called lambda, it is the asymptote of the learning curve.

Question 185

what is the learning ceiling

Answer 185

When dog completely knows that click means treat Asymptote is very much determined by the value of the US (Whether it is good or bad) If it was about shocks, probably a more intense shocks get you to a higher lambda.

Question 186

what is really important in lambda

Answer 186

All about the US — that is the important part of this. Relatively of the US is very important

Question 187

the associative strength of the US and CS is V (y axis) =

Answer 187

predictive value

Question 188

the asymptote is determined by

Answer 188

the value of the US

Question 189

Change in associative strength (formula)

Answer 189

△V = ⍺β(ƛ-V)

Question 190

⍺

Answer 190

CS = ⍺ (it comes first) = salience of CS

Question 191

β

Answer 191

US = β = salience of US

Question 192

V

Answer 192

V = associative strength = learning parameter = how well the CS predicts the US

Question 193

ƛ

Answer 193

ƛ = ceiling effect (how much that animal is going to learn) — stands for the US

Question 194

△

Answer 194

△ = delta = change

Question 195

(ƛ-V)

Answer 195

(ƛ-V) = surprisingness of the US = prediction error

Question 196

V = ⍺β(ƛ-V) is pretty much

Answer 196

the Rescorla Wagner model

Question 197

(ƛ-V)

Answer 197

The difference between the US and what the CS predicts (ƛ-V) is called the prediction error.

Question 198

as trials of CS-US associations accumulate

Answer 198

the CS becomes a better and better predictor of the US.

Question 199

the more pairings of the CS/US...

Answer 199

the easier it is for the animal to learn

Question 200

with time, the prediction error becomes

Answer 200

smaller and smaller

Question 201

classical conditioning =

Answer 201

djusting the prediction error

Question 202

If you have a large (magnitude) US (𝛃), lambda will be...

Answer 202

larger too (higher asymptote)

Question 203

More likely to hit the lambda with...

Answer 203

a strong/salient US

Question 204

So how can you get the blue line in clicker training? (larger magnitude)

Answer 204

- Use a strong US e.g., in dog training, liver treats instead of regular kibble - Easier to make you get nervous about the possibility of getting a shock when the shock is very painful

Question 205

can lambda move?

Answer 205

The lambda can move (you can bypass the ceiling of (1.0), all depending on the strength of 𝛃

Question 206

if the CS is really salient (⍺)...

Answer 206

the learning will be faster (i.e., it affects the rate of learning).

Question 207

how can you get faster learning

Answer 207

get a salient CS (⍺)

Question 208

salient CS example

Answer 208

e.g., in dog training, a stronger, louder, better defined (crispier) clicker sound.

Question 209

strong CS, great US =

Answer 209

lot of learning and fast learning.

Question 210

can boredom occur over time

Answer 210

With time is it possible things get boring, boredom can take place. It is technically possible that after a while these parings get boring. Then you could replace the clicker with a whistle or something to add an element of surprise/novelty.

Question 211

A strong US (liver treats), or 𝛃...

Answer 211

will impact the associative strength/asymptote or lambda (𝛌).

Question 212

A salient CS (crisp click), or 𝛂...

Answer 212

will impact the rate of learning.

Question 213

is it possible for other CSs to be present during conditioning?

Answer 213

yes

Question 214

If 2 or 3 CS’s are present...

Answer 214

they all contribute to the conditioning (and may compete... as you may remember with blocking)

Question 215

how did they add the sum of associative strengths to the formula?

Answer 215

This is why they added sigma to the formula, the sum of the associative strengths — acknowledge the potential of sum variability in adding CS.

Question 216

when might a number of associative strengths be at play

Answer 216

In the acquisition of a traumatic experience it might not always be clear what caused it, but rather an association of many things

Question 217

compound conditioning is also known as

Answer 217

potentiation effect

Question 218

formula with compound conditioning

Answer 218

𝚫V = 𝛂𝛃(𝛌-𝚺V).

Question 219

what does sigma refer to

Answer 219

sum of

Question 220

the Kamin blocking experiment: phases

Answer 220

1. Phase 2: The tone competes with the light 2. Phase 3: The tone loses

Question 221

why does the kamin experiment play out that way

Answer 221

Why? The light already accurately predicts the US (shock). - The light already acquired the strength of the CS/US pairing

Question 222

Medical diagnosis of dietary (GI) intolerance example

Answer 222

Phase 1: Patient drinks milk » stomach ache Phase 2: Patient drinks milk + and has garlic » stomach ache Phase 3: Could the garlic cause the stomach ache? Diagnosis: “Unlikely, eat garlic, but avoid milk”

Question 223

Think of this example: Medical diagnosis of dietary (GI) intolerance -- implication (garlic/milk)

Answer 223

Implication (cognitive): Clearly the physician is thinking of a milk allergy here. But… if she/he had considered that both milk and garlic are common triggers for IBS (therefore a milk — and garlic — intolerance, not a milk allergy), then garlic would have been seen as a potential player. - Have to test things one by one. If you start combining things you will never be able to isolate the cause — not just because of the metabolic response, but also the chance of classical conditioning

Question 224

Suppression ratio

Answer 224

There is less learning with a high suppression ratio.

Question 225

If a new stimulus is presented with an other that was previously conditioned...

Answer 225

then little or no conditioning will occur (resulting in a high suppression ratio).

Question 226

Remember that you initially pair a tone or noise (CS) to a shock (US). Then, the same noise AND a light are paired to the shock (US). the outcome is...

Answer 226

that the response to the light is not well acquired (high suppression ratio).

Question 227

what did the RW model address

Answer 227

Rescorla-wagner explained blocking, big victory. However, it does not explain a lot of other things.

Question 228

how did the RW model explain blocking

Answer 228

By showing that the second stimulus does not acquire an associative value in the second wave

Question 229

how to unblock

Answer 229

modulate the US (saliency of US is the focus of the model)

Question 230

UNBLOCKING =

Answer 230

Increasing the US (shock) in phase 2.

Question 231

neural network explanation

Answer 231

???

Question 232

what decreases during extinction

Answer 232

In other words, V, or the associate strength, will decrease during extinction

Question 233

what happens to lambda in extinction

Answer 233

So lambda becomes 0 (there is no US anymore)

Question 234

what happens when you reintroduce a US

Answer 234

There is very strong spontaneous recovery as soon as you reintroduce the US. What may actually work is if you repair the clicker with the food (marking or keep going signal). Fear is that if you never pair the clicker with the US again, it may change the meaning of the click. The meaning of the click is how it was associated with the US. What if sometimes you click and its not followed by food, but sometimes you click and it is followed by food. That model here does not address that.

Question 235

Conditioned inhibition à la Pavlov:

Answer 235

L (CS) + Shock (US) L (CS) + N (CS) + no US > the noise becomes an inhibitor

Question 236

Protection from extinction

Answer 236

If you combine an inhibitor with an excitor during extinction, the inhibitor will protect the excitor from losing associative strength (V).

Question 237

what does protection from extinction help to explain

Answer 237

This may explain relapse in exposure therapy for fear or phobias — you would try to extinguish the link

Question 238

in exposure therapy for fear/phobias you would...

Answer 238

extinguish excitors. While doing so, you may have cues that become inhibitory.

Question 239

initially during fear conditioning...

Answer 239

it looks like you have no more fear. But if later you test the excitor by itself, then fear comes back.

Question 240

why does that occur during fear conditioning?

Answer 240

- Well remember the concept of surprise. - The noise (inhibitory) predicts no US (no fear). But if you present the light again... fear is back! - Surprise is the key for learning! - In some cases you can have two excitors

Question 241

what happens if you have two excitors?

Answer 241

- Then you have a very efficient extinction!

Question 242

To get very efficient extinction trials...

Answer 242

combine the CS with other excitors (not inhibitors).

Question 243

negative contingency

Answer 243

US is less probable in the presence of the CS - This scenario leads to inhibitory conditioning of the CS

Question 244

neutral contingency

Answer 244

US is equally probable in the absence or presence of the CS - This leads to no learning

Question 245

Anytime you have a CS or US that is conditioned

Answer 245

context is always processed at the same time (comparator theory)

Question 246

Mackintosh and Turner (1971)

Answer 246

How the attentional theory of theory (attentional learning of classical conditioning)

Question 247

mackintosh and turner groups

Answer 247

Group 1: used noise followed by shock, no phase 2, light noise than much better shock, then test light and see what happens Group 2: noise followed by shock, light/noise shock, light noise than much bigger shock, then test

Question 248

what happened in mackintosh and turner experiment

Answer 248

- Based on Rescorla and Wagner, the groups should not have differed, but the control group (1), sees more conditioning (remember, with the SR, low scores indicate more conditioning) - Group 1 (control) learns because of unblocking, i.e., the use of a larger shock. - Group 2 (experimental): The LN - Shock trial interfered with the learning. - Group 2, phase 2: Subjects learn that the light (L) is redundant, so they pay less attention to it!

Question 249

what were mackintosh and turner trying to show

Answer 249

that we learn to ignore redundant predictors of a US, good or bad.

Question 250

what do we learn to do with redundant predictors of a US

Answer 250

learn to ignore them!

Question 251

So it may not be about the ineffectiveness of the ___ (RescorlaWagner), but rather the ineffectiveness of the __ (Mackintosh).

Answer 251

US, CS

Question 252

so how well does the CS predict the US?

Answer 252

If the CS is a good predictor, then we pay attention to it. If not... we don’t

Question 253

Attention paid to the CS

Answer 253

how well the CS predicts its consequences

Question 254

___ is the predictor

Answer 254

CS

Question 255

Mackintosh model is all about the...

Answer 255

attention we pay to the CS

Question 256

the more ___ the ___, the faster the learning.

Answer 256

salient, CS

Question 257

moral of the story

Answer 257

US and CS are important

Question 258

Group dog clicking classes — how does it work if everything you hear in the room is click?

Answer 258

Think about it. How does the dog know which click is theirs? There must be a mechanism for that dog to understand which click is important to them. Attention is important!

Question 259

problems with RW model?

Answer 259

1. Does not explain well extinction of inhibition 2. Does not explain well latent inhibition (pre-exposure to a CS) 3. May have some issues even with the blocking effect 4. Miller, barnet & Grahame identity 23 predictive failures of the model

Question 260

who raised issues with blocking effect of RW?

Answer 260

Mackintosh raised some issues with how the US needs to be surprising, thinks attention is more important

Question 261

mackintosh and turner main idea

Answer 261

The more you repeat something, the better the outcome — talking about elemental isolated stimuli that get a certain type of learned irrelevance

Question 262

mackintosh and turner finding about CS?

Answer 262

CS that is not redundant (or surprising) may be good

Question 263

what is at the forefront of mackintosh and turner model?

Answer 263

CS

Question 264

conclusion of mackintosh and turner?

Answer 264

we learn to ignore redundant predictors of a US - So it may not be about the ineffectiveness of the US but rather the ineffectiveness of the CS

Question 265

so how well does the CS predict the US?

Answer 265

If the CS is a good predictor, than we pay attention to it. If not…we don't

Question 266

cognitive: attention paid to CS <<<

Answer 266

how well the CS predicts its consequences

Question 267

mackintosh model?

Answer 267

- Attention to the CS = alpha in the RW formula - CS salience (alpha) is key, the more salient — the faster the learning - Game is to pay attention to the CSs that are the best predictors of the US

Question 268

what is latent inhibition the same as

Answer 268

CS pre-exposure effect

Question 269

what is latent inhibition/CS pre-exposure effect?

Answer 269

if a stimulus is not suprising anymore, after a while you will stop paying attnetion

Question 270

does the mackintosh model always work?

Answer 270

no. thats the problem

Question 271

the pearce hall model...

Answer 271

changed the landscape of models for classical conditioning

Question 272

pearce hall two groups?

Answer 272

Group 1 = tone and shock, tone and SHOCK Group 2 = light and shock, tone and SHOCK

Question 273

pearce hall results

Answer 273

We know here that group 1 should learn quickly, because the tone is present in phase 1 and 2, with a US of stronger value. What we have here is the opposite due to negative transfer

Question 274

premise of pearce hall model?

Answer 274

Why pay attention to a well established CS? (Elements of mackintosh here) Instead, we pay attention to unknown/unconditioned CS — really coming back with idea of surprise

Question 275

main point of pearce hall

Answer 275

organisms attend to reposed to uncertain predictors

Question 276

pearce hall perspective aligns well with...

Answer 276

modern attentional thoery

Question 277

what is alpha based on in pearce hall model?

Answer 277

As in mackintosh, alpha is attention but the value of alpha is based on how surprising the US is — important because alpha brings attention in

Question 278

what matters in pearce hall model

Answer 278

both CS and US, could argue all elements of RW model are here

Question 279

kaye and pearce study

Answer 279

Step 1: Condition rats to light CS — initially orient to it (eg. Sign tracking). If they orient to it it means they are paying attention Step 2. Three groups Continuous Partial None

Question 280

continuous group?

Answer 280

CS/US pairing on every trial (R&W in support of this)

Question 281

partial group?

Answer 281

50% CS/US paring

Question 282

none group?

Answer 282

CS never paired with US

Question 283

kaye and pearce study results

Answer 283

Found that partial pairing actually worked the best. Continuous pairing is fine but not great, no pairing does not perform with. Partial pairing is really when the rats are paying attention, in other words, a little uncertainty is not bad.

Question 284

conclusion of pearce/hall and kaye/pearce

Answer 284

- How surprising the US is on preceding trial is crucial - How surprising the US is, depends on the CS - Little bit of uncertainty may actually maintain responses - The vale of alpha remains high - US component (R&W), CS component (Mackintosh) — this pairing creates uncertainty and ends up being in your advantage - Explains negative transfer - Explains latent inhibition

Question 285

The Pearce and hall model, under the kay and Pearce experiment reinforces that...

Answer 285

uncertainty is a good thing and both CS/US are important

Question 286

pearce and hall model summary of negative transfer: two groups?

Answer 286

Group 1 = tone and shock, tone and SHOCK Group 2 = light and shock, tone and SHOCK

Question 287

pearce and hall model summary of negative transfer: phase 1

Answer 287

Phase 1: CS becomes a perfect predictor of the US - Problem, US becomes unsurprising with time — the repetition, fact that CS is losing some of its predictive value is problematic. With time, less and less attention is payed to the CS.

Question 288

pearce and hall model summary of negative transfer: alpha value at beginning of phase 2

Answer 288

is very low

Question 289

pearce and hall model summary of negative transfer:

Answer 289

initially little learning is possible - Negative transfer occurs: you pay less and less attention to the CS

Question 290

pearce and hall model summary of negative transfer:

Answer 290

Synthesis is to in a sense, take all of the components together Importance of CS comes together with Pearce and Hall

Question 291

what are we paying attention to? according to pearce and hall

Answer 291

1. Cues with high predictive value (mackintosh) 2. Cues high in uncertainty (pearce-hall): organisms attend to uncertain predictors

Question 292

what does RW suggest (main idea!)

Answer 292

Surprisingness of the US — high saliency of US gets you to pay a lot more attention to predictors

Question 293

attnetion is important but depends on...

Answer 293

previous learning

Question 294

two things we have learned...

Answer 294

can condition attention AND can condition motivation as well. Cognitive processes of attention and motivation can be conditioned, these are essential to learning

Question 295

modern version of short term memory?

Answer 295

working memory

Question 296

who proposed working memory?

Answer 296

werner honig

Question 297

what is working memory?

Answer 297

Memory that you use as you are doing a task. Could include rats in a skinner box, or people being given a phone number. Form of short term memory that stays as long as the info is relevant (usually not very long)

Question 298

how does sensory memory go to STM?

Answer 298

self-generated priming

Question 299

how does LTM go to STM?

Answer 299

retrieval-generated priming

Question 300

what does wagner do in his new model alone?

Answer 300

Brings back the surprise element here Brings back primed component of STM

Question 301

wagner priming (problem?)

Answer 301

If you are primed, in most memory tasks that primed element will be easier to retrieve from memory later. Here, a primed element is less surprising — therefore we have a bit of a problem

Question 302

two types of priming

Answer 302

self generated retrieval generated

Question 303

self generated priming?

Answer 303

via sensory memory (basic structure of the stimulus)

Question 304

retrieval generated priming?

Answer 304

via LTM (actually from a complex retrieval process of info — already most likely in LTM)

Question 305

surprise can be reduced in ___ ways

Answer 305

2

Question 306

retrieval generated priming?

Answer 306

From long term memory via a retrieval cue

Question 307

self-generated priming?

Answer 307

From sensory memory via a recent presentation

Question 308

blocking =

Answer 308

retrieval generated priming

Question 309

retrieval from LTM?

Answer 309

the US is primed by the CS

Question 310

what is priming of the US with the CS?

Answer 310

blocking

Question 311

priming can explain...

Answer 311

blocking - because the surprisingness of the US is dampened

Question 312

continuous learning =

Answer 312

in the moment, self-generated

Question 313

long pause, later CS is presented =

Answer 313

long term memory, retrieval generated priming

Question 314

priming of the CS

Answer 314

- RW: combined processing of the CS and US in STM is important - Wagner thought that maybe he could look into latent inhibition (CS pre exposure effect again) - With this model, exposure to the CS before conditioning should reduce the CS surprisingness.

Question 315

wagner can explain...

Answer 315

habituation

Question 316

wagner model what is habituation

Answer 316

habituation is simply a decrease in the "surprisingness" of the stimulus with both types

Question 317

SOP

Answer 317

Standard operating procedure or sometimes opponent process

Question 318

what kind of model is SOP?

Answer 318

This is a connectionist model, very much a neural network model — parallel processing

Question 319

who massively influenced SOP?

Answer 319

donald hebb

Question 320

what was SOP originally called?

Answer 320

PDP or parallel distributed processing

Question 321

what does PDP entail

Answer 321

1. Processing in the brain is parallel, not serial (like the information processing models always suggested). 2. Processing in the brain is distributed, not localized or modular (like emphasized in [clinical] neuropsychology and evolutionary psychology).

Question 322

Those models (PDP) have been very successful in some areas of cognitive psychology and neuroscience...

Answer 322

1. Learning and memory (including priming) 2. Perception / pattern recognition

Question 323

But (PDP) not so successful in other areas (mostly "higher" cognitive processes):

Answer 323

1. Language 2. Problem solving

Question 324

what does SOP do that is new?

Answer 324

address time

Question 325

SOP (3 features)

Answer 325

1. Addresses the timing of the CS and US 2. Addresses backward conditioning (US presented before the CS). Explains how you can get conditioned inhibition; timing is still important here for this effect. 3. Compound conditioning: Leads to inhibition.

Question 326

AESOP who proposed?

Answer 326

wagner and brandon

Question 327

AESOP adds?

Answer 327

Wagner and Brandon added the emotional qualifies of the stimuli, particularly the US.

Question 328

what does AE mean in AESOP?

Answer 328

affective extension - emotional aspect

Question 329

AE suggests that new nodes are activated by a CS

Answer 329

sensory node affective node

Question 330

result of AESOP

Answer 330

CS gets linked to both of these nodes

Question 331

Potentially every time you have CS/US paring...

Answer 331

you are tapping into the sensory system, likely near the limbic

Question 332

AESOP is the only theory that says...

Answer 332

pairing can be both sensory and emotional

Question 333

what is AESOP useful in discussing

Answer 333

It is particularly useful when discussing US's that may have a strong emotional valence (positive or negative).

Question 334

what can AESOP help to explain

Answer 334

how the emotional response may potentiate the sensory response

Question 335

AESOP and trauma?

Answer 335

So this adds an other variable/factor to conditioning. Think about flashbacks in PTSD! Physical stimuli and emotions being combined

Question 336

Suddenly AESOP is much about the ___ or ____ and is less stuck in a "____" view of the US.

Answer 336

Suddenly this theory is much about the "sign" or "stimulus" and is less stuck in a "poverty of the stimulus" view of the US.

Question 337

what is also responsible for poverty of stimulus?

Answer 337

Cognitive psych is also responsible for poverty of stimulus because it focuses on representation of the stimulus by itself. Here we are saying that none of that is terribly valid if you don’t take the emotional value into consideration.

Question 338

what is the parallel processing dimension of AESOP?

Answer 338

simultaneous sensory and affective processing.

Question 339

what does AESOP explain

Answer 339

Explains why conditioning works better (often worse), with strong reactions — eg. Trauma

Question 340

potentiation between sensory and affective suggests that..

Answer 340

limbic system is working with sensory areas of brain

Question 341

CS in AESOP

Answer 341

They (Wagner & Brandon) were influenced by Pearce (from Pearce and Hall)

Question 342

pearce came up with?

Answer 342

configural thoery of learning

Question 343

what is configural learning

Answer 343

The set, or configuration, is what is conditioned to the US

Question 344

what is elemental learning?

Answer 344

Each CS conditioned independently to the US

Question 345

pearce configural theory of learning?

Answer 345

configural theory of learning — means that if you have many CS, Pearce saw them as typically being a whole. In other words, they are more than just the parts (elemental view)

Question 346

the configurational perspective is ___ in nature

Answer 346

gestalt

Question 347

configurational perspective gestalt

Answer 347

It means that the whole is more than the sum of its parts. Example: Play a chord on your piano or guitar, say A minor. - Means that configurations are meaningful and you can actually be conditioned by configurations

Question 348

So compound stimuli, especially if in the same sensory modality, are...

Answer 348

more that the sum of the elements.

Question 349

So just adding different CS's just...

Answer 349

weakens the value of the single, elemental CS (they are in fact, literally not the same).

Question 350

From a neural network theory point of view, and consistent with SOP, if you condition ABC as CS's, but then just test AB, you would ...

Answer 350

lose 33% of the conditioning (from R-W, remember that CS's share a lambda).

Question 351

order of progression of theories

Answer 351

Wagner's original theory ~ SOP ~ AESOP

Question 352

AESOP explains

Answer 352

- Compound conditioning effects (Rescorla-Wagner) - Attentional and priming effects (Mackintosh, Pearce-Hall) - Time constraints in conditioning - Sensory and emotional dimensions of conditioning - Stimulus configuration, and even generalization

Question 353

operant conditioning, focus on...

Answer 353

Focus on response-outcome learning

Question 354

applications of operant conditioning

Answer 354

- Parenting - Training (animals, humans, etc.) - Teaching (behaviourist methods) - Cognitive Behavioural Therapies (and others), behaviour modification - Programming of gaming software, gambling games and machines (e.g., slot machines) - Advertisement - Any social situations (making compliments, ignoring people, etc) - The tipping game! Etc. (so many more examples)

Question 355

differences between classical and operant conditoning

Answer 355

- Contingency between outcome and response is extremely clear in operant — the is why an animal learns - In classical conditioning this is very fuzzy because it is often automatic/unconscious

Question 356

similarities in classical and operant conditioning

Answer 356

- Both have a learning curve. - Both have extinction.

Question 357

thordnike theory

Answer 357

Thorndike: Reinforcement theory » law of effect.

Question 358

guthrie theory

Answer 358

Contiguity theory » Contiguity is enough.

Question 359

guthrie contiguity examples

Answer 359

(Why the dog doesn’t learn to hear the porcupine? Because of lack of contiguity in time and space) Often dogs get hit by porcupine, may feel little bit of pain but there is an analgesic in the quills. Very quickly the animal doesn’t feel pain, the more they start moving the more the quills go in. Why is it that my dog is not learning the porcupine is a big thing from the initial hit? Because of continuity. They end up learning the most pain they experience that day is hours later at the vet clinic. That is what they learn to be afraid of.

Question 360

stimulus sampling theory

Answer 360

stimulus elements are at the centre of this theory:

Question 361

Learning curves show an improvement because...

Answer 361

the organism notices more stimulus elements.

Question 362

for guthrie, a reinforcer is...

Answer 362

just a very salient stimulus — we know it is more than that now. How close time and space associations are happening is the key for Guthrie

Question 363

tolmans approach

Answer 363

One of the first cognitive theories (uses the term mental map or cognitive map).

Question 364

for tolman behaviour is...

Answer 364

inherently flexible.

Question 365

tolman rats main point

Answer 365

He showed that rats in mazes learn better about places than responses Main point: They can learn on non-reinforced trials. It becomes obvious when the reward is provided. This is called latent learning

Question 366

learning is not...

Answer 366

performance!

Question 367

the role of the reinforcer is to...

Answer 367

provider motivation (idea developed by spence and hull)

Question 368

tolmans approach order of variables

Answer 368

SD > R > O SD = discriminative stimulus R = response O = outcome

Question 369

discriminative stimulus gives...

Answer 369

a response for a specific outmode (Eg. Kibbles in spinner box) — operating conditioning. Contingencies must be vert explicit.

Question 370

thorndike box SD/R/O values

Answer 370

SD = box R = sequence of events to open O = outcome

Question 371

for thorndike and guthrie ____ associations

Answer 371

SR

Question 372

for tolman ___ associations

Answer 372

SS

Question 373

rewards thorndike idea?

Answer 373

they reinforce

Question 374

rewards guthrie?

Answer 374

they are a very salient stimulus

Question 375

rewards tolman/hull/spence?

Answer 375

they motivate

Question 376

what do rewards actually do (consensus)

Answer 376

In fact they are/do all three — reinforce, salient stimuli and actually motivate.

Question 377

two types of priming

Answer 377

positive and negative

Question 378

positive priming

Answer 378

pre-exposure to stimulus facilitates received of that stimulus

Question 379

negative priming

Answer 379

Wagner SOP the exposure to the stimulus gets in the way of learning

Question 380

negative transfer

Answer 380

when any type of prior knowledge interferes with current knowledge (learning)

Question 381

pearce-hall negative transfer

Answer 381

Peace-Hall negative transfer: pairing a CS with a weak US slows-down conditioning when the CS is subsequently paired with a strong US

Question 382

classical view of dog bit situation

Answer 382

CS is dog, US is bite

Question 383

AE-SOP view of dog bite situation

Answer 383

CS is still dog, now there is two US — one purely sensory (pain), one affective (learned fear)

Question 384

AESOP explains ____ very well

Answer 384

AESOP explains conditioned emotional responses, often based on fear conditioning, very well

Question 385

contiguity

Answer 385

all about spatial and temporal (how close in space and time things are) - CS co-occurs with the US: they are contiguous, or close together, in space and time

Question 386

contingency

Answer 386

the CS predicts the US: the occurrence of the US is contingent on the prior occurrence of the CS.

Question 387

two main methods of a skinner box

Answer 387

discrete trial method free-operant method

Question 388

discrete trial method

Answer 388

Stand-alone trials, and many of them, in a session. This controlled by the experimenter. - Eg. Gadbois lab — presenting dog with a problem. Accumulate trials

Question 389

free-operant method

Answer 389

The animal or the human control the apparatus/computer. - Typically used for skinner box — animal controls flow of experiment by giving specific responses to specific stimuli

Question 390

free operant set up

Answer 390

Typical learning curve as response rate showing acquisition and extinction (when you stop producing the kibbles). — response for minute The cumulative recorder taking the cumulative responses. As you learn they increase, as it extinguishes they decrease

Question 391

SD → R → O

Answer 391

Able to identify SD as something that should provoke a response and outcome

Question 392

S△ → R → ØO

Answer 392

The other stimulus, the one you don’t want to respond to because it won’t provide an outcome

Question 393

Sdelta or?

Answer 393

S-

Question 394

Operant way of teaching discrimination (nose hold in position) example

Answer 394

Eg. Want a dog that can find specific snakes (eg. Gartner snake) in a park that is full of different kinds of snakes. Need dogs to tell difference between ribbon and garter snakes. How do you teach a dog to discriminate just ribbon snakes? We rely on their nose. It is very simple, show the ribbon snake smell and encourage the dog for sniffing (give treat or click). Now grab the other stimulus (garter snake) and do not encourage the dog for sniffing (no treat). Do you train no response (just walk away) or a distinct response for a no (train them to sniff, then sit down because it is wrong). S+ or yes response would be hold nose. Contrasting approach: systematically reinforcing a response where they initially just show interest and ignore with the S-.

Question 395

concept of discriminative stimulus

Answer 395

Light on » Press lever » get food Light off » Press lever » no food So after a while, the rat presses the lever only or mostly when the light is on.

Question 396

what does it mean when an operant response is brought under stimulus control

Answer 396

a stimulus will set the occasion for the response. - The light is a discriminative stimulus (also labelled S^D or S+): It is associated with reinforcement. The other stimulus is the S delta or S-.

Question 397

occasion-setting in discriminative stimulus is...

Answer 397

operant

Question 398

____ is necessary for all sorts of operant learning

Answer 398

stimulus control

Question 399

shaping, autoshaping and ____ behaviours according to skinner

Answer 399

superstitious

Question 400

superstituous behaviours example

Answer 400

If there is too much of a delay before giving reward, can occur because they have to hold their nose for 5 seconds, you can develop superstitious behaviours. Smarter breeds are more likely to develop superstitious behaviours (border collies) — because they are trying to determine what you want. Idea is that you need to extinguish the response, may feel like you are going backwards. You have to get them to step back, do a normal entrance, then reinforce. How do you do this? Often going backwards. Get them to enter, put them right in front of the vial before the trial starts and reshape just the nose hold — not the whole entrance. This helps to reduce the possibility of reinforcing the wrong behaviour.

Question 401

habit slips

Answer 401

In maze running, food encountered in the maze won't stop the running to the goal (the end that is baited). - If you put the reinforcement before the end goal, often the rat will just run by it and complete the task. That just means they have really learned the task and developed the habit of going through the full actions (almost the opposite of superstitious behaviours in some ways)

Question 402

Habit slips, or strong discriminative stimuli »

Answer 402

response associations are not uncommon.

Question 403

habit slip example

Answer 403

- Going to pick-up somebody at the airport but you take the Dartmouth Crossings exit because you go there often. - These behaviours can be really hard to extinguish

Question 404

the ______ of the response, does not matter if you focus on the goal

Answer 404

specificity

Question 405

lashley showed...

Answer 405

that rats that learn to run dry maze, will immediately learn to swim it if it is flooded.

Question 406

can all dogs trained in the lab transfer to the field

Answer 406

no only about 50% - dog knows the odours but the change in context is enough to confuse them

Question 407

transfer of learning is a form of

Answer 407

generalization

Question 408

specificity of response errors in change of context

Answer 408

might have to do with focus of control, or sign tracking/goal tracking

Question 409

EDT

Answer 409

errorless discriminative training

Question 410

EDT Terrace example

Answer 410

The dog learns to respond to the S+ (e.g., scent of lavender). The S- (scent of oregano) is introduced early. The S- is not presented in the first trial, and then is presented in weak form at first, and gradually strengthened: this is “fading” (fading-in).

Question 411

attentional dimension of EDT

Answer 411

Manipulation of the saliency of the S-. Difficult to do if you don’t have the right equipment

Question 412

EDT is all about

Answer 412

what you are getting the animal/human to pay attention to. Based on simple premise — during learning you commit the least amount of error possible and also the outcome is that (often) the performance of the animal/human will be with less errors than with typical discrimination training.

Question 413

EDT example (long)

Answer 413

S+ (one you want the dog to respond to) is aways shown at full strength. You present the two stimuli at the same time. S+ (lavender) always shown at full saliency, but not the oregano. Present a very weak, varying, amount of the S- (oregano). Sometimes they will look towards the oregano, but ignore that and they typically go towards the more salient stimulus. Idea is that through all the trials, you make the saliency of S- stronger until they are at the same level. This works because you are indicating right away to the dog which stimulus is important and which to ignore. In some cases, they barely learn to ever respond to the S- so by the end of the task they have responded only to the S+, even though the S- is at full strength at the end.

Question 414

why is EDT considered to be errorless

Answer 414

you diminish the amount of potential error that can be created

Question 415

EDT tried to be used with what population

Answer 415

This got the attention of people working with a autistic children. If you go with trial/error method they often get very frustrated when it is not reinforced. How to teach autistic children without them making errors, created techniques that resembled this in some ways.

Question 416

main advantages of EDT

Answer 416

Large reduction of errors compared to traditional discrimination learning Less mistakes are possible during training. No negative emotions during training (e.g., frustration, helplessness, etc.) The training is potentially very fast

Question 417

EDT conclusions

Answer 417

Mistakes are NOT necessary for learning! Evidence from neural networks that sometimes it is It is likely an attentional mechanism: You learn (early!) to ignore the irrelevant stimulus and basically never respond to it (or at very low rate and only initially in the training) Trial and error is NOT necessary, in principle you can learn without the error just by focusing on attentional processes right from the get go

Question 418

main disadvantages of EDT

Answer 418

Modifications or reversals in training are difficult: e.g., if lavender (S+) becomes the S- and you try to train the S- (oregano) as the S+. Acquisition of the S+ will be delayed and/or impaired. Example: Training a dog to discriminate (for field search and detection) a specific species of snake within the Thamnophis genus. If you ever want that dog to switch to Garter snakes as the target species, it may be challenging.

Question 419

___ of dogs wont train through EDT easily

Answer 419

2/3

Question 420

examples of how classical conditioning is involved in operant conditioning

Answer 420

Money — there is a delay, humans still understand this The clicker in animal training or TAG teach (TAG = Teaching with Acoustical Guidance; e.g., used to train movement precision in dance or sports) — make the exact moment they did the exact right movement

Question 421

the clicker is a ______ stimulus

Answer 421

classically conditioned discriminative

Question 422

the clicker is a _____ for food (US)

Answer 422

pavlovian CS

Question 423

why is the clicker used in operant conditioning

Answer 423

because it has acquired a reinforcing value in itself

Question 424

where did the idea of using just the clicker come from

Answer 424

Situation arises first when trying to train orcas and dolphins in captivity — idea is that you dont want to interfere with animals during training. How do you tell them “yea you got that right, continue”. The conditioned reinforcer was created, not giving a primary but marking it with a secondary (the clicker or a whistle) That in itself is literally telling you that the clicker does not need to be followed by the reinforcer, at least every time.

Question 425

history of conditioned reinforcer

Answer 425

When pigeons were in the Skinner box, when being shaped and giving the right response, the food magazine would drop but make a click while doing so. Quite accidentally, researchers noticed that when the magazine would empty, pigeons would continue to respond for a long time, and most likely we responding to the click as a conditioned reinforcer.

Question 426

what are conditioned reinforcers now used in

Answer 426

Used a lot with kids, eg. Stickers. These are secondary reinforcers — they have been conditioned to come with a reinforcer later.

Question 427

CS/US link may ____ over time if you stop paring the click with food for animals

Answer 427

deteriorate

Question 428

how to use the clicker two steps

Answer 428

1. Pair the clicker sound with the primary reinforcer , e.g. food (with a short of a delay between the two as possible). 2. When that pairing is well established, then:

Question 429

when there is both a clicker and food...

Answer 429

the clicker announces the food

Question 430

what happens when you use the clicker in isolation

Answer 430

In this case, the clicker announces the possibility of food. "... as long as the secondary reinforcer is occasionally followed by food, the behaviour is maintained” Or the clicker is used as a marker (yes, this was the right answer, but continue what you are doing; e.g., in “chaining” — more on this below). Could be called “keep going signal” Becomes useful in a number of situations — solves issue of distance, need for spatial/temporal precision,

Question 431

why use the clicker

Answer 431

1. Necessary because of delay in delivering the primary, usually because of distance between the trainer and the trainee. 2. Spatial and/or temporal precision is necessary. 3. You are training behavioural chains, and use the clicker as a "keep going" signal (similar arguments as per #1):

Question 432

chaining

Answer 432

You use chaining to train complex sequences

Question 433

chaining is common in

Answer 433

sports, music, dance, and other high motor skill learning — eg. Flying an airplane

Question 434

backward chaining

Answer 434

Backward chaining: 3, then 2, then 1 (reverse order of actions). Learn the whole sequence by starting at the end and go backwards. Could be useful in correction of superstitious behaviour in a dog for example.

Question 435

chaining is often use to...

Answer 435

train new pilots and flight simulators

Question 436

"Stimuli associated with primary reinforcement can strengthen behaviour because these stimuli acquire....

Answer 436

their own reinforcing value”

Question 437

conditioned (secondary) reinforcers and motivation

Answer 437

If not that, then at least it is feedback that informs the organism it will soon get the primary (sometimes called a “ keep going signal" in training circles).

Question 438

still the idea that ____ is what gets ____ to happen

Answer 438

Idea that learning/dopamine is what gets attention/learning to happen

Question 439

negative contrast

Answer 439

Rats (and humans) are sensitive to the contrast between reinforcers of different values.

Question 440

negative contrast three rat groups

Answer 440

Group 1: plain water as reinforcement — fine, especially if animal had been water deprived. But water is not a terribly good reinforcer. Food when hungry works better than water when thirsty. Group 2: Sweetened water as reinforcement — glucose gives it more value Group 3: Sweetened water then plain water — they will get less responses to plain water.

Question 441

is reinforcement or punishment better

Answer 441

Science suggests that reinforcement is the way to go, but that punishment works (albeit with negative consequences) if just talking about performance, punishment does work. Question is what supports reinforcement — it is ethical. Important to make an ethical argument, what ethics tell us is that you well develop a better relationship with the human/animal if you use reinforcement as opposed to punishment

Question 442

punishment works to suppress a behaviour, but at a cost...

Answer 442

Emotional distress, frustration, toxic relationship, etc. And not even necessary... other things work.

Question 443

Concurrent reinforcement to punishment may still keep the behaviour:

Answer 443

Sam is misbehaving at school and gets punished by the teachers... but the attention he gets from his peers is more reinforcing.

Question 444

When the contingency are figured-out, it may lead to cheating:

Answer 444

till producing the "bad" behaviour when the punisher is not around.

Question 445

best technique of reinforcement/punishment

Answer 445

DRA or differential reinforcement of alternative (other) behaviours

Question 446

From a ethical perspective people say to always go with

Answer 446

positive reinforcement — that is better because it won’t damage relationship with the person. 98% of the time this is the ethical and scientific perceptive

Question 447

is timing important for...

Answer 447

conditioned reinforcers

Question 448

timing is particularly important for

Answer 448

early training — kibble has to come immediately after the behaviour you want is produced

Question 449

continuous reinforcement

Answer 449

every time the animal produces the behaviour it is reinforced.

Question 450

if you want a behaviour to be easily extinguishable reinforce it _____

Answer 450

continuously. - Behaviour that is reinforced every time, when you stop reinforcing it will extinguish immediately. Typical example in parenting — you give a reward to your child for doing something, one day you dont and they won’t do the behaviour anymore.

Question 451

continuous reinforcement sets up...

Answer 451

an expectation

Question 452

When you initial train an animal (or human for that matter), you typically start with

Answer 452

a continuous reinforcement (CRF) schedule.

Question 453

types of intermittent reinforcement

Answer 453

Fixed ratio Variable ratio Fixed interval Variable interval

Question 454

how are intermittent reinforcement/schedule of reinforcement recordered

Answer 454

Those schedules are recorded by a cumulative recorder and the output is a cumulative record. It shows the pattern of response based on the reinforcement delivery (the schedule).

Question 455

ratio schedules

Answer 455

Ratio is based on a count of the number of occurrences or trials — each trial is every chance you would have to reinforce the organism. In a ratio schedule, you do not reinforce every trials — instead, skip a few then reinforcer - Ratio schedules are based on the rate of response, i.e., a x number of responses (e.g., you reinforce the rat every 5 good responses, this would be a FR 5).

Question 456

interval schedules

Answer 456

Interval is about the timing that elapses before the reinforcement, not the number of trials. - Interval schedules are based on an interval of time between reinforcements, i.e., regardless of the number of correct bar presses (assuming some good responses occurred), the rat is reinforced at an interval of time (e.g., every 2 minutes, this would be a FI 2).

Question 457

what kind of reinforcement were dog trainers taught to use

Answer 457

Dog trainers were taught to do continuous reinforcement, because ‘otherwise the dog would bark or get frustrated’ — gadbois says if that is happening you are doing it wrong

Question 458

what is important for intermittent reinforcement

Answer 458

Intermittent reinforcement for every experiment people were taught to go slowly. Eg. If you see any stress or aggravation in your rat you are going too quickly. Observing your organism is important — figure out what they like or dont. If you sense any frustration stop because something is wrong.

Question 459

95% of the time when you start applying intermittent reinforcement and you get frustration it is because...

Answer 459

you are going too fast

Question 460

study using intermittent reinforcment with dogs

Answer 460

Study said dogs in intermittent reinforcement got more stressed — they went from 100% continuous reinforcement immediately to 60%. Imagine your dog gets constantly reinforced and then suddenly only 60% of the time and they wonder why the dog is stressed. Thats because you were going way to fast. Actual way to do it is very slowly omit a few of the rewards, with time increasingly omit more. It is that simple. If you start to see any aggression, reinforce a little more. You are setting up the animal to understand that when they give a right answer — give them feedback they did, but not always a reward. Eventually you will have pigeons that will give 3000 responses without reinforcement — that is how powerful intermittent reinforcement is when used right.

Question 461

what is needed for the transition from continuous to intermittent reinforcement

Answer 461

need an adaptation period

Question 462

fixed ratio

Answer 462

- FR 5 = every five answers you give a reward - If you are continuously reinforcement start with FR2 for example, needs to be low early on. - FR 5 is extremely predicable — they know when the reinforcement is coming. They will tap the lever five times quickly to get the reward, they will accelerate. - They have an expectation and that is the issue with the fixed ratio — they learn they just have to produce five responses

Question 463

variable ratio

Answer 463

- Variable ratio = VR 5 +/- 2 - You set the computers to come up with these numbers, what it means is that they know they will get a response but not every time.

Question 464

fixed interval

Answer 464

- FI 10 - Unit of time - Regardless of number of responses they give, they will not be reinforced until the interval is over — eg. 5 minutes - Now they know if they wait a certain time they will get a reinforcement - Problem is that they figure out what they do doesn’t really matter until the end of the internal - Can correct this problem using variable interval

Question 465

variable interval

Answer 465

- VI 10+/- 3 minutes - Dont know exactly the interval of time where they will start getting a reward — therefore will work really hard in those conditions - Will typically get you a more consistent reinforcement, lower chance of extinction

Question 466

what methods of reinforcement is preferred

Answer 466

Anticipation and learning connected, anticipation will be better when you dont know when you are getting the reward therefore fixed and variable intervals are better.

Question 467

psyches of dog trainers now are wrong

Answer 467

Most dogs don’t care about the treat because they are so into the task (working breeds). Whole point of doing intermittent reinforcement is that you train your brain that this is fun because you dont know when you are getting the reward — its a surprise! Very resistant to extinction — falls into category of habit learning When you keep habit learning it becomes pretty much unextinguishable. Problematic when you do therapy on behaviours you are trying to get rid of that have become habits.

Question 468

problems with fixed ratio or interval

Answer 468

they are predictable!

Question 469

what is the benefit of variable ratio or variable interval?

Answer 469

By applying those, the rat or pigeon is unsure about when the reward is delivered and is more likely to respond eagerly and the behaviour acquired is much more resistant to extinction

Question 470

real life is...

Answer 470

intermittent

Question 471

what relationships are hard to snap out of

Answer 471

Relationships that are hard to snap out of are relationships that are unpredictable, they’re exciting — there is a lot of anticipation That is because you are on a variable ratio — most social interactions are on intermittent reinforcement Reinforcement matters in social behaviour, that means that all principles you know from learning theory applies to these situations.

Question 472

fixed ratio (example and consequence)

Answer 472

Reward every xth response. Consequence: Produces a good amount of behaviours, but lots of post reinforcements pauses. You know that after you are done that ratio, there is no need to rush.

Question 473

variable ratio (example and consequence)

Answer 473

The ratio is harder to predict. Consequence: High and persistent rates of responding. - Slot machines!

Question 474

fixed interval (example and consequence)

Answer 474

Reward at a fixed (xth) interval of time. Consequence: Low rate response after each reward. Same problem as fixed ratio because it is relatively easy to predict.

Question 475

variable interval (example and consequence)

Answer 475

Steady rates of behaviour because the outcome is difficult to predict in time. Social media likes or messages

Question 476

variable interval: social media

Answer 476

social media companies have people that are trained in learning theory to control this, they control when they let you know, how they let you know, when you stop responding and they look at your rate of responses. They use these intermittent schedules of reinforcement, mostly variable ratio/interval Can be used for good and bad things!

Question 477

Variable ratio/interval gives...

Answer 477

more persistent behaviour less chance of extinction

Question 478

Ratio schedule

Answer 478

Strong correlation between the rate of reinforcement and the rate of behaviour. Faster response rates (the “hurried-up" nature of the behaviours produced, e.g., lever-pressing).

Question 479

Interval schedule

Answer 479

Reinforcement rates can be steady, but the rate of responding can vary immensely. More pauses and wait times between responses.

Question 480

compound schedules

Answer 480

Use of more than on schedule. In fact, outside of the laboratory, these are by far the most common schedules of reinforcement.

Question 481

different schedule examples

Answer 481

Mixed/multiple/tandem/chained

Question 482

matching law

Answer 482

When training an organism you want to see what they respond to best — test what works best. There is some evidence that some species will not deal well with variable intervals (as much as we said they are good if you go slow enough), some species will not work unless it is continuous reinforcement - Skinner thought there were universal laws here but that is not the case

Question 483

matching law example

Answer 483

Give two different schedules to a pigeon A: pecking at disk A is reinforced on a VI 2 min schedule B: pecking at disk B is reinforced on a VI 1 min schedule - In this experiment you get more for VI 1 min, this could change with time and will depend on how hungry the animal is. Eventually, they may switch to VI 2.

Question 484

relative rate of responding to a particular choice (response alternative) will match (equal) the...

Answer 484

relative rate of reinforcement for that choice (response alternative).

Question 485

what does matching law explain the relationship between

Answer 485

explains the relationship between payoff and choice Idea is that animals will try to maximize reinforcement at a lower cost

Question 486

what explains the matching law

Answer 486

1. Animals are likely to try to maximize the rate of reinforcement: How much you get per unit of time / session. 2. They try to improve the local rate of reinforcement, so they shift between the options (choices). This is called melioration 3. Rate is not the same as strength, so both are factors (the same way as speed and mass influence momentum in kinematics): This brings us to momentum

Question 487

melioration

Answer 487

this depends on emotional state — will often simply switch to a better schedule when it fits their current metabolic needs better

Question 488

what is reinforcement a test of

Answer 488

persistence rather than competence

Question 489

delayed reinforcement

Answer 489

ometimes reinforcement is delayed. How well animals take delayed reinforcement matters (if too long, organisms do not take it well). If humans are being explained to — going to work two weeks for a lot but then get paid, humans are okay with that.

Question 490

Impulsiveness vs self-control (impulse control)

Answer 490

impulsive people are more likely to want their goods right away

Question 491

procrastination and working priorities

Answer 491

a lot of this has to do with your ability for delayed gratification. Idea is that you have ease into it and develop a technique of finding small gratifications.

Question 492

implications of reinforcement

Answer 492

1. delayed reinforcement 2. impulsiveness vs. self control 3. procrastination and working priorties

Question 493

Example of tasks testable with people and animals: Choice between a large delayed reward vs. a smaller immediate reward.

Answer 493

The general observation: Animals and humans are impulsive, i.e., prefer smaller immediate rewards.

Question 494

example of delayed reinforcers

Answer 494

biweekly paycheque

Question 495

delayed reinforcers are...

Answer 495

not as reinforcing as immediate ones.

Question 496

If you are trying to intervene with things that have regular schedules it can be challenging — really hard to get people to adhere to things unless you have ...

Answer 496

a pre-commitment strategy

Question 497

what do pre commitment strategies have applications or

Answer 497

studying, weight loss, cessation of smoking, snoozing the alarm in the morning, etc.

Question 498

what are pre commitment strategies

Answer 498

get people to announce what they are going to do (eg. New years resolution). You are dangling the possibility of shame if you dont go through with what you said you would Example: Dry January and other resolutions that are “publicly" announced.

Question 499

what are factors in delayed gratification

Answer 499

temperament / personality, development and psychopahtologies

Question 500

personality example

Answer 500

impulsivity

Question 501

development example

Answer 501

Impulse control (especially in boys), etc.

Question 502

psychopathologies example

Answer 502

DHD, etc. ADHD is complicated: idea that attention may be a problem. If you cannot focus attention, learning is not going to happen.

Question 503

strategies for delayed gratification

Answer 503

1. Make the immediate reward less "rewarding" (or even coming at a cost). 2. Make abstention profitable: You get a reward if you don't procrastinate. 3. Make the associated delayed reward explicitly less appealing if the choice is to get rewarded immediately

Question 504

Make the immediate reward less "rewarding" (or even coming at a cost).

Answer 504

Put a cost on engaging in the behaviour now. They still get a choice in the matter.

Question 505

Make abstention profitable

Answer 505

Ironically, that profit can be delayed, it will still work, e.g., "If you finish your homework now, you can play video games for longer tomorrow”. Appetitive reinforcement?

Question 506

Make the associated delayed reward explicitly less appealing if the choice is to get rewarded immediately

Answer 506

Personality difference here, impact may depend on whether you can be patient.

Question 507

what can help delayed grat

Answer 507

Much of this can work if you make "anticipation" exciting (as it should be). It tends to be more rewarding to some than others. "Waiting" does not have to be aversive. Humans have working memory and prospective memory (planning) — ability to deal with things that haven’t happened yet.

Question 508

making anticipation exciting depends on

Answer 508

- Working memory: Good working memory = reduced impulsiveness. Distraction from the appeal of the immediate reward is the strategy here. Remember that working memory is conceptually overlapping with attention. - Prospective memory and planning: To process delayed rewards, you need to be able to process the future (i.e., mental "time travel")

Question 509

what are Impulsiveness / sensation-seeking / risk-taking implicated in

Answer 509

asal ganglia/dopaminergic system

Question 510

Personality traits that may modulate and influence behaviour in both humans and animals. Many of these traits are associated with pathologies or risky behaviours:

Answer 510

Gambling Addiction Risk junkies (sky-diving, bungee jumping, risky sexual behaviours, etc) ADHD Borderline personality disorder — characterized by risky sexual behaviours

Question 511

dopaminergic system dogs 1992 study

Answer 511

Dogs in 1992: paper that showed a massive difference between breeds that have high vs. Low dopamine. High = border collie, jack Russel, Australian Shepard (more energetic) Low = great dane, saint bernard, Newfoundlander (more lazy) Difference is activity level and impulsiveness Can be applied to friends: you may have friends that prefer skydiving vs. Friends that prefer laying on couch

Question 512

We now know through neuroimaging that there is a correlation between...

Answer 512

raits and dopaminergic system — idea of addictive personalities. We know this is a thing, but we don’t know what to do about it. We think the key to additive personalities is sensitivity to reinforcement — responding to variable intervals, excitement, etc. Idea that it is dopamine quantity and sensitivity both — unknown

Question 513

day-to-day applications for managing delayed grat

Answer 513

1. Give choices — with any organism, test your procedure, reinforcement and schedule — every individual may be different 2. Help make better choices 3. Be clear about the impact of the choices — explain consequences

Question 514

clinical approaches for managing delayed grat

Answer 514

1. Contingency management — directly from learning theory 2. Incentive-based treatment

Question 515

Behavioural economics and neuroeconomics (not huge on exam)

Answer 515

- These two fields have massively influenced research in behaviour and neuroscience, and have been influenced by those sciences as well. Go a little beyond basic cognitive psych and neuro — weird interaction between rational and irrational behaviour - We know that humans love to think we are cognitive and rational, good at following rules. When actually, we are unbelievably limbic in the way we make our decisions. What we do is limbic not cortical, based on emotions/raw motivations/reinforcement — applies to psych, bio, etc.

Question 516

theories of reinforcement

Answer 516

1. Drive reduction theory 2. The Premack principle 3. Behavioural regulation theory 4. Selection by consequences

Question 517

Interestingly, Skinnerian (radical behaviourism) theories are ______ in explaining reinforcement.

Answer 517

not very helpful

Question 518

skinnerian reinforcment

Answer 518

1. They avoid mentalistic concepts, hypothetical/theoretical constructs, e.g., “motivation”. — had to adopt one to explain how motivation works (drive theory)The Skinnerian arguments to explain reinforcement are circular arguments (tautologies). 2. They do not have a theoretical framework to explain reinforcement, nor do they seem to mind.

Question 519

Clark Hull and drive reduction theory

Answer 519

theory of attrition/deficit, missing something so you get it — reducing drive

Question 520

Drive reduction theory:

Answer 520

Reinforcers reduce drives.

Question 521

drive reduction theory example

Answer 521

Example: Treats reduce hunger (while training a dog; logically, this means that a hungry dog would learn better, faster, etc.).

Question 522

drive reduction theory is a ____ theory

Answer 522

It is a deficit theory (you do or learn something because you need to reduce a drive, e.g., hunger, thirst, etc.). Hull eventually realized the drive theory had limitations and started to talk about “incentives". In the section on motivation, we will cover the incentive theory that does a much better job at explaining motivation and its role in learning (and behaviour in general). Neuro and psych have adopted incentive theory — works better than drive because you can classically condition motivation

Question 523

premack prinicple aka

Answer 523

the differential probability principle

Question 524

classical/traditional principle

Answer 524

Contingency between a behaviour and the reinforcement.

Question 525

premack principle

Answer 525

The contingency is between two behaviours — contrast between pressing a lever and getting fed, to him this is the essence of reinforcement. More probable response will reinforce less probable responses

Question 526

premack prinicple in clear terms

Answer 526

more probable responses will reinforce less probable responses The question: What is the contingent behaviour? In the Skinner box, what are the two behaviours?

Question 527

skinner box: premack

Answer 527

Skinner box: Bar pressing Eating - But the rat would rather eat... In other words, reinforcement happens when the (instrumental) behaviour gives access to a preferred behaviour. - The bar pressing gets you to eat which is the motivation - Theory about contrasting two different processes

Question 528

solution for reinforcement!

Answer 528

Do a preference test! See what is more reinforcing to the subject: Food, toy, play, etc. Find the behaviour (instrumental act) that provides access to a more preferred behaviour.

Question 529

premack 1959

Answer 529

play a pinball machine or have candy Some kids chose candy (they spent more time eating candy than play pinball). With these kids, if candy is made contingent on playing pinball, pinball playing increases. The reverse is true for kids that chose pinball over candy.

Question 530

preference is...

Answer 530

really important!

Question 531

preference lab example

Answer 531

Dog owners bringing dog preferred treats to lab

Question 532

Behavioural regulation theory (Timberlake et al): premise

Answer 532

al behaviours have a preferred level. If you restrict access to that behaviour, the organism will engage in an other behaviour or behaviours to gain back access to that behaviour.

Question 533

Behavioural regulation theory: staddon

Answer 533

Bliss point and the minimal distance model. How the organism distributes its behaviours to maximize reinforcement (the bliss point). So all organisms will learn to maintain a preferred distribution of behaviours.

Question 534

behavioural regulation theory main assumption

Answer 534

The main assumption of this model is that humans and animals will seek and attempt to maintain an optimal combination of activities (the bliss point). Idea is that as organisms we try to meet our metabolic needs

Question 535

Selection by consequences (Donahoe et al.): Premise

Answer 535

Learning is like natural selection.

Question 536

selection by consequences, premise is like...

Answer 536

"weeding-out" (Bouton’s term) or "pruning" (my term) by reinforcement (or punishment...).

Question 537

"ingredients" for natural selection

Answer 537

1. You need a trait to select. Here it is a behaviour. 2. You need a selective agent. Here it is reinforcement (via the teacher/trainer/computer).

Question 538

selection by consequences mechanisms are similar to...

Answer 538

imilar to natural selection — reinforces the theory that learning is not just about acquiring information, but also pruning information

Question 539

selection by consequences prerequisites

Answer 539

1. variation 2. fitness consequences 3. mode of inheritance 4. limited resources

Question 540

variation

Answer 540

This is now variability in forms of behaviours, across situations, etc.

Question 541

fitness consequences

Answer 541

This is now simply reinforcement contingencies

Question 542

mode of inheritance

Answer 542

The learning process itself (operant conditioning).

Question 543

limited resources

Answer 543

The constraints on the process, and deprivation (e.g., of food) is the analogy. - eg. If you are reinforcing with food, maybe you need hunger. Some dogs are suggested to skip breakfast, so that they will work harder to increase the drive

Question 544

learning is...

Answer 544

ontogenetic (it happens during a lifetime, i.e., it is a developmental process)

Question 545

learning happens to...

Answer 545

an individual

Question 546

natural selection is...

Answer 546

phylogenetic (happens over long periods of time and across generations, i.e., it is an evolutionary process).

Question 547

natural selection happens to...

Answer 547

species

Question 548

selection by consequences elements found in connectionist theories

Answer 548

1. Neural networks models make similar assumptions (pruning process are described). 2. Genetic algorithms and genetic programming: They use explicitly biological terms such as "selection", “mutation", "crossover", etc

Question 549

motivation is a...

Answer 549

hypothetical construct

Question 550

potential issues with Structuralism, Functionalism, Gestaltism

Answer 550

Potential issues: Introspection, lack of experimental rigour.

Question 551

radical behaviourism (skinnerian)

Answer 551

Potential issues: No hypothetical constructs or intervening variables tolerated (i.e., mediators, e.g., memory, emotions, motivation, the brain).

Question 552

radical behaviourism (skinnerian)

Answer 552

Potential issues: No hypothetical constructs or intervening variables tolerated (i.e., mediators, e.g., memory, emotions, motivation, the brain).

Question 553

Operational behaviourism (modern academic behaviourism).

Answer 553

Potential issues: Still focussed on conditioning to explain all behaviours; and focus on behaviour (as opposed to its underlying processes).

Question 554

Cognitivism: Focus on thought processes, awareness, consciousness, etc.

Answer 554

Potential issues: Poverty of the stimulus and representationalism.

Question 555

Connectionism

Answer 555

- PDP models, a.k.a. neural network models - A form of neural associationism - Explains well perception, learning, memory, but not so well the higher cognitive processes.

Question 556

What is motivation? (4)

Answer 556

1. Hypothetical (theoretical) construct 2. Need for intervening variables » Hull » Operational behaviourism (Tolman) 3. So important that Woodworth thought psychology should be called “Motivology". 4. But why did some behaviourists (mostly Hull and Tolman) see the need for motivational concepts? -- The main reason: Explain how reinforcement (and punishment) work.

Question 557

Where is the evidence for motivation?

Answer 557

behavioural neurophysiological

Question 558

behavioural evidence for motivation

Answer 558

1. Behaviour is variable — some days you are horny, other days not — this changes how you feel 2. Behaviour is persistent — why do we procrastinate? Can persist in good and and behaviours regardless of the outcome

Question 559

neurophysiological evidence for motivation

Answer 559

1. The impact of physiological states relating to homeostasis (balance, a compensating system), allostasis (systems do not balance easily because there are many things going on in parallel — chaotic and paradoxical), metabolism 2. Neurotransmitters: e.g., dopamine (one of the most important NT for motivation, that is why it has a direct impact on learning) 3. Hormones: Mostly peptide and steroid hormones

Question 560

How was motivation historically defined?

Answer 560

1. ethology/biology 2. psychology

Question 561

Ethology/biology historical definition

Answer 561

drive as instincts — the hunting drive is not really a desire, it is more an incentive behaviour

Question 562

psychology historial definition

Answer 562

1. Drive theory: a response to deficits (it is a "deficit theory", focus on -R) — deficit theory, works well with homeostasis, weak theory in some ways 2. Incentive theory: focus on +R, intrinsic motivation, etc — much more based on positive reinforcement and classical conditioning

Question 563

incentive theory motivation

Answer 563

1. Incentive motivation: Motivation for instrumental behaviour created by anticipation of a positive reinforcer. Also called the rG-sG mechanism (from Hull & Spence) - incentive motivation is about how you acquire a reinforcer 2. Incentive learning: A process by which organisms learn about the value of a specific reinforcer while they are in a particular motivational state - Learning of how to be motivated, you can teach an animal to be motivated — requires not always making things too predictable or too easy

Question 564

do you need reinforcement for learning?

Answer 564

Technically no. incidental learning (latent learning), implicit learning dont require reinforcement

Question 565

first demonstration of incidental learning

Answer 565

Tolman and latent learning

Question 566

are reinforcers necessary for learning

Answer 566

Reinforcers are not necessary for learning but are there to motivate behaviour and give purpose. Reinforcers affect motivation, not learning. Learning is not performance… but motivation is performance.

Question 567

will learning always extinguish if not reinforced

Answer 567

Nope Eventually extinction will happen in classical conditioning and operant learning Habit learning: operant w a lot of repetition, everything gets autonomized and the acquisition of habits will not get extinguished

Question 568

Habit learning…

Answer 568

- Habit learning was hard to situate in learning theory - Hull and Lashley (e.g., “Maze running habit”) used the term. - Or Tolman’s “habit slip”. - The modern use started with Hirsh (1974), then Miskin et al. (1984).

Question 569

concept of habit learning is very close to...

Answer 569

"skill learning” (both with a motor component) and both are seen overall as “implicit” or “non declarative” forms of learning

Question 570

habit learning and skill learning both seem to be controlled by

Answer 570

same parts of brain — basal ganglia, dopamine, subcortical Idea you can go beyond conditioning and get really strong learning that is not much at the mercy of external reinforcement because it has become intrinsically motivated Eg. Dogs in field don’t always need a treat, sometimes just motivated enough by finding a turtle The more you do something, the more likely you are to develop intrinsic motivation Difference between liking and wanting something

Question 571

types of declarative (Explicit) learning

Answer 571

facts and events

Question 572

types of nondeclarative (implict) learning

Answer 572

skills and habits, priming, simple classical conditoning, nonassociative learning

Question 573

Habit learning is:

Answer 573

Instrumental in nature, and very much associated with “motor learning” and the basal ganglia (as opposed to hippocampal-based learning). - More subcortical than we would imagine

Question 574

Hippocampal (contextualizer) learning

Answer 574

Context learning (spatial and temporal). Defined as “rapid learning”. We also know from Gray that the hippocampus is a “comparator”. The hippocampus gives context (the what, the when and the where). Change in context can make you lose some learning or ability to adjust in a certain environment

Question 575

Basal ganglia-based learning

Answer 575

Motor in nature. Defined as “slow learning”. Now often defined within the realm of the “cortico-striatal loop” (two loops, one is motor, the other is motivational) If a rat learns to run the maze, but can do it by swimming it is not just basal ganglia earning (memorizing specific motor sequences), but also hippocampal. They both work.

Question 576

The cortico-striatal loop (from Seger & Spiering, 2011)

Answer 576

- Not just cortical or subcortical, but both -So motor-based learning (habit and skill learning) and motivation are linked in the brain. -Both are heavily driven by the dopaminergic system. -Remember: Both "motor" and "motivation" (even “emotion”) have the same etymological root. --> Why you give amphetamines to people with ADHA (ritalin)

Question 577

All parts of the brain are kinda connected...

Answer 577

if you look at the etymology but motor and motivation go together

Question 578

the evolution of the concept of motivation: Hull

Answer 578

drive theory

Question 579

drive theory

Answer 579

- Hull, early work: Reinforcement » reduction of drive (e.g., hunger) - Behaviour strength = D x H - Behaviour strength = Drive (need) x Habit (learning) - Interaction between drive and habit - Behaviour strength = Need x Learning

Question 580

Resistance to extinction depends on:

Answer 580

1. Degree of food deprivation (how hungry you are): Drive or D 2. How much prior reinforcement (reinforcement history) you got: Habit or H

Question 581

Incentive theory: Hull take 2

Answer 581

- The concept of incentive motivation: Hull & Spence - Behaviour strength = D x H x K Behaviour strength = Drive x Habit x Incentive Behaviour strength = Need x Learning x Incentive Realized the K explains better forms of learning — innate

Question 582

Incentive =

Answer 582

motivational effect of reward

Question 583

Hull’s theory (even the update with Spence) is seen as

Answer 583

outdated today

Question 584

Now we conceptualize motivation as

Answer 584

response to an anticipation of need

Question 585

Tolman’s view

Answer 585

Tolman: He argued that animals learn expectancies Food rewards are confirmatory (or not). Food = catalyst (Rescorla, 1985).

Question 586

for a strict, radical behaviourist

Answer 586

- the operational terms “anticipation” and “expectancy” are problematic... but only theoretically. - So they prefer still the term “reinforcement” - Clearly the reward value in a learning task is contingent on how easy it is to get it - The way the see the reinforcer is as a confirmer (yea, I was right — like tap teaching in dance)

Question 587

Here we lose the radical behaviourists because

Answer 587

(anticipation is involved) The physiological processes are also not a focus of radical behaviourism. The processes are invisible to the naked eye... unless you have technology... and we do in neuroscience. Although the process is classically conditioned... Indeed, the motivational effects of reward come from classical conditioning (this is “incentive learning”). This is what incentive learning really is

Question 588

The motivational effect of reward comes from

Answer 588

classical conditioning

Question 589

reinforcement affecting motivational state idea

Answer 589

Idea is simple: if you are hungry, you are learning about that state of being hungry. It is actually when you are hungry that you are open to the incentive value of the situation you are in. Only when you are hungry can you learn, because it sets you up for that anticipation. Anticipation itself becomes the reward (remember the clicker!) Clicker announces food > clicker announces possibility of food > pigeon that is still pecking at same stimulus despite the fact there is no food This occurs because there is still anticipation! In a sense, motivational states are occasion setters

Question 590

incentive learning

Answer 590

-You learn in the hungry state about the incentive value -So you learn about an incentive, and then: -You link this to the instrumental action. -If you experience the +R in a non-hungry state, then you will have no interest in the +R (e.g., food). -Motivational states are occasion setters! (Davidson, 1993, 1998, etc.) or facilitators (Rescorla, 1985) -Great review of these ideas in Dickinson & Balleine, 1994

Question 591

Interlude; what are occasion setters?

Answer 591

Safety signals are negative occasion setters: "if this light is on, the shock won't happen" "if I have my pills, I won't have a panic attack" and even: "if I have my pill bottle -- even if empty -- I won't have a panic attack".

Question 592

We can have modulation in conditioning

Answer 592

Facilitation and facilitators; occasion setting and occasion setter

Question 593

occasion setters are...

Answer 593

environmental cues (CS's) that "set the occasion” for conditioning to occur. They "facilitate" the conditioning.

Question 594

Occasion setting (Holland) =

Answer 594

facilitation (Rescorla)

Question 595

Occasion setters (Holland) =

Answer 595

facilitators (Rescorla) » motivational states

Question 596

An o.s. is a

Answer 596

CS that confirms a CS-US pairing.

Question 597

An o.s. is a

Answer 597

modulator of the conditioning between a CS and a US. In itself, it becomes rewarding

Question 598

So we have three ingredients in classical conditioning modulation:

Answer 598

1. The CS 2. The US 3. The modulator (second CS) = occasion setter = facilitator » motivational states

Question 599

what does the modulator add

Answer 599

This adds what we call "conditional relations" and "conditional control”. It is not unlike basic discriminations in operant conditioning when we reinforce the S+ (the target stimulus or "good response"), and not the S- (the foil, distractor, or "wrong response").

Question 600

pavlovian discrimination

Answer 600

Two ingredients: 1. The target stimulus: The CS that is present on every trial in a CS-US pairing. It is the target because that is the one that the subject responds to. 2. The feature stimulus: The CS that is present only in trials indicating if the US occurs or not (depending on the procedure). On top of operant discrimination, you can add something from a specific situation using classical conditioning

Question 601

occasion setter facilitates...

Answer 601

a specific association between stimuli — facilitator, can be there or not but its presence facilities something

Question 602

So back to incentive motivation and learning:

Answer 602

-Hunger triggers the foraging behaviour (system). -Motivation "energizes" the action. -But you must learn ("know") that the action leads to the reinforcer. -Occasion-setters (or facilitators) help in doing so! -You must learn that the reinforcer has a positive effect on the motivational state. -The motivational state increases the desirability of the reinforcer

Question 603

"Learning will always extinguish if not reinforced”

Answer 603

Actually extended training makes behaviour less sensitive to its consequences » habit learning (e.g., Holland 2004).

Question 604

It is not about the need, or the reward...

Answer 604

but about the anticipation of the reward…

Question 605

acquired motivation

Answer 605

Motivation is not from drives/needs, but rather the anticipation of need... Back to Tolman and latent learning... 1.Exploration of the maze. No reward. 2. Reward in the maze: They excel, with no previous reinforcement history! So... rewards: 1.Reinforce behaviour 2.Motivate

Question 606

early behaviourists believed in

Answer 606

gradual learning, not one trial learnings. But if a rat can do it just by exploring and no reinforcement, there is something else going on. Rewards reinforce and motivate behaviour

Question 607

tolmans ultimate contribution

Answer 607

As Bouton (2016) says elegantly, "a reward at the end of the tunnel" goes a long way Sometimes just the possibility of it is enough Flaherty (1996) discuses at length these phenomena in “Incentive relativity".

Question 608

crespi (1942)

Answer 608

Negative and positive. contrast effects

Question 609

crespi study

Answer 609

An increase in reward (1 to 16 pellets) results in an increase in running speed and elation = positive contrast effect. -- Acquire the task, at shift time there is no change. The learning is stabilized. A decrease in reward (256 to 16 pellets) results in a decrease in running speed and depression = negative contrast effect. -- Huge crash in running speed, they adjust their behaviour based on what they are reinforced in 1 pellet to 16 pellets -- Huge jump in their learning! Post-shift is huge

Question 610

crespi study rationale

Answer 610

Those effects are caused by a change in the expected value of the reward. Not really the reward, but what you expect the reward to be. How much effort you are putting in might be modulated by the expected value of the reward. Not doubt this is paradoxical

Question 611

A few words about paradoxical reward effects:

Answer 611

Rewards can sometimes weaken behaviours (in an instrumental context) - -Eg. You get tired Non-rewards can sometimes strengthen behaviours (in an instrumental context) -- Eg. Allude to incentive/intrinsic motivation This includes the negative contrast effects mentioned above.

Question 612

Rewards may not always be as magical as people think, they often and quickly become irrelevant

Answer 612

- Animal trainers seem to forget this — the importance to develop incentive motivation and that you do not need to always reward. - Gen-X thought always rewarding was a great idea — everyone gets a trophy. Sounds good but it creates more problems than anything else, namely increasing a dependency on extrinsic motivation.

Question 613

What about extinction and motivation?

Answer 613

Extinction has a lot to do with the magnitude of the reinforcement. The bigger the reward, the bigger the extinction The higher magnitude for a reward, the faster the extinction. (Hulse, 1958; Wagner, 1961)... UNLESS, you are dealing with HABIT LEARNING

Question 614

over learning extinction effect

Answer 614

Many rewarded trials can increase the rate of extinction (as opposed to fewer rewarded trials)

Question 615

what should you do with high value reinforcement

Answer 615

Maybe make it intermittent

Question 616

example of intermittent reinforcement

Answer 616

Slot machines are very careful in giving very big payments very rarely. The small payments keep you there, the inconsistent big prizes are the anticipation.

Question 617

Can rewards have negative effects?

Answer 617

Extrinsic rewards can affect performance negatively. "Punished-by-reward" principle in humans

Question 618

"Punished-by-reward" principle in humans

Answer 618

People like this do not want to take responsibility and are also extremely sensitive to external rewards. When it is not rewarding anymore they disconnect. This is the danger of using extrinsic motivation too much — idea is trying to find ways of developing intrinsic motivation

Question 619

can verbal rewards be good?

Answer 619

For some people these are more important than money Expectations, overall, are important.

Question 620

Partial Reinforcement Extinction Effect (PREE) and PERSISTENCE

Answer 620

- Rescorla, 1999 (and a few other studies) — said connection between US/CS was really important He realized he might be wrong after Pearce/Hall and Wagner -Partial reinforcement reduces the associative change produced by non- reinforcement. Two groups of rats: CRF (continuous reinforcement) and PRF (partial reinforcement) In acquisition, CRF (100% of trials are reinforced): better (faster, stronger acquisition) -But: In extinction, PRF (e.g., 50% of trials are reinforced) » persistence

Question 621

running speed (Effort for motivation) study

Answer 621

Running speed (effort for motivation) — in acquisition trials the continuous reinforcement were doing better than the partial. Seeing a little less effort In extinction: the continuous lose the behaviour much more quickly, running speed goes down dramatically. The partially enforced rats see effort go down over time, but it is much slower.

Question 622

working dogs hpoglycaemia struggle

Answer 622

Train dogs to recognize signs of anxiety in people with PTSD Biomedical alert and the lack of maintenance training — people with hypoglycaemia in people with diabetes This events are very raw, so it is hard to reinforce. Even when we have people that volunteer to produce those samples, well over have the time they forgot. When they get hypo they deal with that situation at hand, not about training the dog. The dogs then do not get reinforced. That is problematic unless if during the training they start using partial reinforcement. Real life for these dogs means that dogs will not always get reinforced, but they need to keep detecting it. Performance collapse, in part because of rare events: Few chances for reinforcement... if the responses to the events are even reinforced when they occur! Learned industriousness (Einsenberg, 1992) >> this can transfer to other tasks.

Question 623

Sequential theory (Capaldi)

Answer 623

How to explain PREE? Two theories: 1. Frustration theory (e.g., Amsel): Not well supported 2. Sequential theory: Not about frustration 1. Extinction = many non-rewarded trials (by definition!) 2. No problem! Especially if this matches acquisition done in PRF! 3. It is the memory of the acquisition phase that "sets the tone" for what to expect. Set expectations early and your future behaviour is dependent on this. This is why always reinforcing is a bad idea. What happens during the acquisition phase is what matters.

Question 624

the sequential theory, and small and large rewards: Capaldi and Capaldi (1970), Leonard (1969)

Answer 624

Sequences of: non-reinforced trials (N) reinforced trials with large rewards (R) reinforced trials with small rewards (r). Two acquisition types: rNR RNr rNR is more resistant to extinction as suggested by the sequential theory This is because they expect less initially

Question 625

is frustration theory very well supported?

Answer 625

no, not very well supported. Frustration does not seem necessary to justify motivation. In fact, some cognitive factors seem at least as important: - Memory - Associative strength

Question 626

How foraging works example

Answer 626

Important point. Basically said, why is it that there is so much resistance from some behaviourists and some animal trainers on the idea that reinforcing all the time is not the best. They point out that if you actually look at how animals get reinforced in the wild, that reinforcement every time would collapse very quickly Looking at success rate in predation: for most is very low Yet they continue doing it If they were right that you need to be rewarded every time you try to hunt, most lions after a few trials would quit Says: hey, our brains are wired for not knowing and instead hoping. That incentive/intrinsic motivation and effect of anticipation and expectancy.