# Lecture 12 - Threats to Validity Flashcards

1
Q

Define ‘internal validity’

A

The extent to which the results obtained are a function of the variables that were systematically manipulated

• Are changes in the DV caused by changes in the IV?
• Does this study measure what its intending to measure (the IV)

High internal validity - strong evidence of causality
Low - little or no evidence of causality

2
Q

How can we maximise internal validity?

A

Must be able to rule out the possibility of confounding variables (factors other than the IV producing a change in the DV)

Therefore there must be a high amount of control and elimination of potential exteraneous influences

Easiest to do in a highly controlled, laboratory setting

3
Q

Name 6 threats to internal validity

A
```history effects
maturation effects
mortality
instrument decay
participant selection
statistical regression to the mean```
4
Q

What are history effects and how do we solve them?

A

Events occurring during the experiment that are not part of the treatment - eg a fire alarm goes off at 11am and affects all groups in at that time but not all groups in later or earlier conditions

Solution: hold experiences as constant as possible except for the IV. this includes randomly assigning conditions to time - you don’t want all of condition 1 to be in the 11am group!

5
Q

Wha tare maturation effects

A

processes within participants that may simply change due to the passing of time - such as fatigue, hunger, and in the long term, ageing.

6
Q

What is mortality and how do we solve it

A

not neceesarily death! Could just be people who drop out.

The people who drop out may well be different from those who do not, invalidating the study

So this is why we tend to provide an inconvenience allowance, so people are unlikely to drop out

7
Q

What is instrument decay and how do we solve it?

A

can be equipment - becomes less accurate with age

or experimenters - become more skilled with time or become more bored with time

Solution: randomise allocation of experimenters to time slots, train the staff, calibrate instruments regularly

8
Q

What is participant selection and how do we solve it?

A

eg only males in control group but both genders in experimental group - obviously going to affect the result

Solution: random allocation or matched pairs

9
Q

What is statistical regression to the mean?

A

going back to the mean after extreme behaviour - so if someone was selected for treatment because of their extremely low score on a depression test, they may well regress to the mean naturally and you cant tell if this is due to the treatment or just regression to the mean

10
Q

What effects occur that equalise groups when participants are allowed to communicate?

A

Diffusion of treatment effects

Compensatory rivalry - participants who become aware theyre in the control group work harder to overcome this ‘disadvantage’ - also called the John Henry effect

Compensatory equalisation - experimenters aware of conditions make up for what they feel is a disadvantage, eg control group, less effective treatment, and perform things above what they are supposed to do.

11
Q

What effects occur that separate groups when participants are allowed to communicate

A

resentful demoralisation - if control group is aware they are the control group then they may give up and not try very hard, increasing the difference between the groups (but not as a result of the IV)

12
Q

What is external validity?

A

For an IV - Does it accurately measure the phenomenon?

For an experiment - does it represent the real world? e.g. field experiments (but often comes with a lack of control)

13
Q

what is population validity

A

the extent to which the results can be generalised from the experimental sample to the population as a whole

14
Q

what is ecological validity

A

the extent to which the results can be generalised from the set of environmental conditions in the experiment to other environmental conditions

15
Q

Name 4 threats to external validity

A

reactive effects of testing
reactive effects of experimental setting
selection-treatment interaction
multiple treatment interference

16
Q

what is reactive effects of testing?

A

it occurs when a pre-test affects participants sensitivity to the treatment

often happens with studies involving self report measures of attitude

17
Q

What are reactive effects of experimental settings?

A

the conditions of the study are such that results are hard to replicate in non-experimental settings

• Hawthorn effects
• Novelty effects
• Experimenter effects
• Rosenthal effect
18
Q

What is selection-treatment interaction?

A

the possibility that some characteristic of the participants selected for the study interacts with some aspect of the treatment

such as prior experience, learning, personality

19
Q

What is multiple-treatment interference

A

when participants receive more than one treatment,p revious ones may influence subsequent ones

eg sequence effects
carryover effects

20
Q

How can we improve external validity?

A

replication

replication with extension

21
Q

What are the two major threats to statistical validity

A

making a type 1 error

making a type 2 error