Chapter 11: Confounding & Obscuring Variables

Question 1

Potential internal validity threats in one-group, pretest/posttest designs

Answer 1

Maturation
History
Regression
Attrition
Testing
Instrumentation
+combined threats

Question 2

Maturation Threat

Answer 2

Change in behaviour that emerges naturally over time
E.g. boys are rowdy at first in camp cabin, but settle in after a week (also reduce sugar during this time)
Prevention: include comparison group

Question 3

History Threat

Answer 3

Change in behaviour result from historical or external factor that systematically affects most members of the treatment group at the same time as the treatment itself
E.g. boys are rowdy at first in camp cabin, but settle in after a week (also reduce sugar during this time) - but also start a swimming program at this time which tires them out
Prevention: include comparison group

Question 4

Regression Threat

Answer 4

Statistical concept called regression to the mean
When group average is extreme at one time, the next time it is likely to be less extreme
E.g. get perfect score on one exam, more likely to regress or do worse on second exam
Only occur when a group is measured twice and when the group has extreme score on pretest
Prevention: comparison group (regression would occur in both groups), comparison group equally extreme at pretest

Question 5

Attrition Threat

Answer 5

When attrition is systematic and a certain kind of participant drops out
Attrition/mortality: reduction in participant numbers that occurs when people drop out before the end
Can happen when pretest and posttest are administered separately
E.g. the rowdiest camper leaves after day 1, causing the group to appear less rowdy overall
Prevention: remove dropouts’ scores from pretest & posttest data, check pretest scores for extremity

Question 6

Testing Threat

Answer 6

Specific kind of order effect when a change in the participants result of taking a test more than once
E.g. students perform better on posttest in educational settings (not because of educational interventions)
Prevention: abandon pretest, use alternative forms for pretest and posttest, comparison group to rule out effect of repeated testing

Question 7

Instrumentation Threat

Answer 7

When measuring instrument’s reliability/validity changes over time
People who code behaviours are measuring instruments, their judgments might change over time
Different measures are more likely to yield certain results
E.g. campers didn’t become less disruptive, the leader just became more lenient
Prevention: posttest-only design, ensure pre & posttest measure are equivalent, establish reliability/validity of both tests, retrain coders throughout experiment, counterbalance test versions

Question 8

Instrumentation vs. Testing Threats

Answer 8

Instrumentation: measuring instrument has changed
Testing threat: participants change over time from being tested before

Question 9

Combined Threats

Answer 9

Selection-history threat: outside event or factor affects only those at one level of independent variable
Selection-attrition threat: only one of experimental groups experiences attrition
E.g. only extreme depressed patients drop out of treatment group (but not control group)

Question 10

Internal Validity Threats In Any Study

Answer 10

Observer Bias
Demand characteristics
Placebo effects

Question 11

Observer Bias

Answer 11

Researchers’ expectations influence their interpretation of the results
Occurs in studies with behavioural dependent variables
E.g. Dr. expects patients with depression to improve in treatment, so she reports that they do
Comparison group doesn’t correct this

Question 12

Demand Characteristics

Answer 12

When participants guess what a study is about and change behaviour in the expected direction

Question 13

Controlling for Observer Bias & Demand Characteristics

Answer 13

Double blind studies: neither participants nor evaluators know who is in treatment and comparison group
Masked design: participants know what group they are in but observers do not
(Especially important when rating behaviours that are difficult to code)

Question 14

Placebo Effects

Answer 14

When people receive a treatment and improve but only because they believe they are receiving a valid treatment
Used often when participants are given medications
Placebo effects reduce real symptoms (physical and emotional)
Double blind placebo control study: neither patients nor people treating them know who is receiving the real vs placebo treatment (still include no treatment comparison)

Question 15

Null Effect

Answer 15

Independent variable doesn’t make difference or CI includes 0
Could mean that the independent just doesn’t affect the dependent (never know for sure)
Could have a true effect that was no detected in the study due to experimental design feature obscuring it

Question 16

Reasons For Null Effects

Answer 16

Weak manipulation of IV
Insensitive measures of DV
Ceiling and floor effects in either DV or IV
Manipulation checks help detect weak manipulations, ceilings, and floors
Design confounds acting in reverse

Question 17

Floor Effect

Answer 17

All the scores cluster at the low end
Can be the result of a problematic independent variable (e.g. all money amounts are squeezed towards floor)

Question 18

Ceiling Effect

Answer 18

All scores are squeezed together at the high end

Question 19

Noise/Error Variance/Unsystematic Variance

Answer 19

Too much unsystematic variability within each group can cause null effects
Less variability within groups narrows the CI and effect size is larger

Question 20

Measurement Error

Answer 20

Human/instrument factor that can randomly inflate or deflate a person’s true score on dependent variable
E.g. height measurement depends on angle of the person’s eyes who is measuring

Question 21

Power

Answer 21

Likelihood that a study will return an accurate result when independent variable really has an effect
Aspect of statistical validity