Quantitative- threats to internal validity Flashcards Preview

Research 2 > Quantitative- threats to internal validity > Flashcards

Flashcards in Quantitative- threats to internal validity Deck (22)
Loading flashcards...
1
Q

7 threats to internal validity

A
  1. history
  2. maturation
  3. testing
  4. instrumentation
  5. selection bias
  6. attrition (morality)
  7. regression to the mean
2
Q

History

A
  • NOT what happens to the participants before they have participated in the study (not personal histories)
  • it is an event that happens between the pretest and posttest that will mess with results of posttest
3
Q

Maturation

A
  • people change over time, participants mature

- eg. test anxiety in first year vs third year is going to be affected by maturation

4
Q

Testing

A
  • the mere fact of taking the pretest changes things for posttest
  • eg. you ask how people relax and go through methods- this influences people to change their behaviour
5
Q

Instrumentation

A
  • change in the way you measure from pretest to posttest
  • eg. different IQ tests at pretest and posttest
  • 2 different measures of intelligence we don’t know if they are exactly equivalent. We wont know if this has to do with change of test. Do research into the difference between the two tests- assess before you do the experiment
  • If instrumentation changes you introduce a confound- wont know if due to change in instrumentation or genuine change in participants
6
Q

Selection bias

A
  • is when there is an inequality between the pretest and posttest in groups.
  • a control group only works if they are like the experimental group
  • Eg. Test anxiety treatment programme if we have all the anxious people in experimental group, there will be a difference between control and experiment group- will look like experiment didn’t work. -Randomisation is NB to get away from selection bias.
7
Q

Attrition

A

-also called study mortality (people drop out of the study)
- It is an issue if dealing with drugs in study. Some will die, some will just drop out. -Problem with attrition: if all attrition all occurs in experimental group- for example: if testing anxiety in test takers the experimental group will maybe take lots of tests to try reduce anxiety. BUT for very anxious people this is unappealing so they drop out. The experimental group then only contains the less anxious people.
=You can bias your results

8
Q

Regression to the mean

A
  • scores are distributed around the mean
  • you can catch people in an unusual point when scores are unusually low or high and when you next test them they regress to the mean
9
Q

How to control threats to validity: history

A

if both control group and experimental group exposed to same historical event. then they will both change in same sort of way. History is controlled for automatically by having a control group.

10
Q

How to control threats to validity: maturation

A

if your control group and experimental group come from the same population group (eg. First years) all exposed to same environment. The only difference between them is the treatment. A control group controls for maturation.

11
Q

How to control threats to validity: testing

A

if pretest changes behaviour then the control group will also change. Then maybe a test is all that is needed to change behaviour and the treatment isn’t necessary (cheaper)

12
Q

How to control threats to validity: instrumentation

A

-Best way to control is to not change your measure- some circumstances you do have to though. Eg. You may use research assistants and they will each interact with people in a different way which may influence results. Way to manage it: research assistant interviews participant 1 for pretest and again for posttest and research assistant 2 interviews participant 2 for pretest and again for posttest.
When you do need to change measure: using a particular iq test- practice effect. You need to do a study to see if one results differ from another

13
Q

How to control threats to validity: selection bias

A

randomise them using a large sample. No chance for bias. Controls for things you might not think to ask about effects of people’s subjectivity

14
Q

How to control threats to validity: attrition

A

cope by saying we have 100 people at pretest and 6 drop out, you compute data for them – make up data as if they did posttest- statistical way to predict what their answers were likely to be. – called an attempt treat analysis

-when people drop out they create selection bias
-If they died you assume the worst case scenario- that the drug didn’t help
=So if they died because of the drug you cant ignore them

15
Q

How to control threats to validity: regression to the mean

A

solve by randomisation

16
Q

Solomon four-group design

A

exp group 1: pretest, treatment, posttest
exp group 2: treatment, posttest
control 1: pretest, posttest
control 2: posttest

Pretest asks about mindfulness etc. and that the pretest makes people think about the treatment- and they go home and practice it. This is a threat to external validity because your treatment only works in presence of pretest and so it only works in context of your experiment- this is how we control for it (solomon 4 sqr design).
Only use it for threats to external ability. Or test acting in treatment in some way. If pretest works alone then cont 1 will also show change.

If your pretest has an effect then exp 1 and control 1 will show reduction in anxiety at posttest. And exp 2 and control 2 will have higher levels of anxiety at posttest.

Need a very big sample bc you have 4 arms to your study

17
Q

Control problems in experimental research: 2 basic designs

A
  1. between-subjects

2. within-subjects

18
Q

Between-subjects designs

A
  • Used when the IV is a subject variable (e.g., extrovert/introvert; marital status)
  • When experience gained in one level would make it impossible to participate in another level

Two ways to cope with having equivalent groups:
Random assignment
Matching

19
Q

Random assignment

A

goal: to take individual factors that could bias the study, and spread them evenly throughout the different groups of subjects
- not a guarantee
- works best with large numbers

20
Q

Matching

A
  • Useful when you only have a small number of participants available
  • Or the matching variable correlates with the DV (i.e., is expected to affect the outcome in some way)
  • Or there is some reasonable way of measuring participants on the matching variable

MAIN USE:
1. WHEN YOU CAN’T RANDOMLY ASSIGN
SMALLE SAMPLE GROUP AVAILABLE

21
Q

Within-subjects designs advantages

A
  • need fewer people
  • no problems with equivalent groups
  • Reduces error variance, since you have no between-condition individual difference – so it gives more statistical power to find an effect if there is one
22
Q

Within-subjects designs: problems

A
  1. practice effects:IQ improves if you carry on taking tests.
  2. Fatigue effects- performance declines over the day. Need to keep questionnaires as short as possible.
    3.Carry over effects/order effects- if you drink OJ before brushing teeth= fine, swap= gross. The order matters.
    fix this by randomly assigning them to an order= called COUNTER BALANCING