l 5 Flashcards
(30 cards)
Experimenter biases
Biases resulting from experimenter behaviour Often but not always in support of the hypothesis • Recall: Clever Hans
Participant biases
Biases resulting from human behaviour Can support or refute the hypothesis, or have nothing to do with it at all
Participants know they are in an experiment
– We often want to have this experimental realism
– Hawthorne Effect
• Generally, people want to be good subjects
– Orme’s (1962) meaningless task
• Participants are active agents, who want to
discover what they’re supposed to do
– other participants may not care as much…
Controlling participant bias
In a single-blind design, the participant does not
know which condition they are in
• To go further, one may need deception
– e.g. placebo groups, unrelated instructions
• A manipulation check can help detect biases
Expectation effects:
biases introduced by the
desires and expectations of the experimenter
Bio-social effects:
biases resulting from the
experimenter’s appearance (e.g. age, race)
Psycho-social effects
biases resulting from
the experimenter’s personality (e.g. friendliness)
Situational effects
biases resulting from environmental factors (e.g. location, time)
Controlling experimenter bias
Automate the procedures as much as possible
– remove the experimenter, or
– establish testing protocols
• In a double-blind procedure, neither the
experimenter nor the subject know which
condition each subject is in
Independent group designs
- Manipulated independent variable
- Random assignment to create groups
e.g. Mueller & Oppenheimer (2014). The Pen Is
Mightier Than the Keyboard: Advantages of
Longhand Over Laptop Note Taking.
Mueller & Oppenheimer (2014)
NOTES BY HAND > LAPTOP
65 students watched one of five TED Talks after
being given laptops (disconnected from Internet) or
notebooks and told to take notes as they usually do
• After 30 mins, they completed a test containing
both factual questions (e.g., “Approximately how
many years ago did the Indus civilization exist?”)
and application questions (e.g., “How do Japan and
Sweden differ in their approaches to equality within
their societies?”)
In experiments 2 and 3, they completed
conceptual replications of their own study
• Expt 2: telling laptop users not to take verbatim
notes did not change the results
• Expt 3: giving everyone a chance to review their
notes before the test did not change the results
– Ecological validity!
Matched group designs
- Manipulated independent variable
- Matching followed by random assignment to
produce equivalent groups
• e.g. Kroeger, Schultz, & Newsom (2007). A
Comparison of Two Group-Delivered Social
Skills Programs for Young Children with Autism
Kroeger, Shultz, Newsom (2007)
25 children were assigned to either the direct
teaching group or the play activities group.
• The direct teaching group used a video
modeling format to teach play and social skills
• The play activities group engaged in
unstructured play during the sessions.
Children with autism can vary in the severity of
their symptoms, severity could influence the
effect of the manipulation, and the sample is
small, so random assignment may produce an
autism-severity confound
• Children in each group were matched using their
Autism Quotient score
– Pairs of children with similar AQs were created, and
randomly split into the two groups
This study was double blind so the observers
scoring the videos did not know which group the
children were from
• More than one observer scored each video so
that the inter-rater reliability could be measured
Using control groups
Any time you use a manipulated IV, you can use
a control group to compare to baseline
– e.g. if we give participants 3mg, 6mg, or 9mg of some
medication, we can only compare between those
• With a placebo control group, we can measure
baseline when participants think they are
receiving a treatment
• With a no-treatment control group, we can
measure baseline when they think they are not
– A wait-list control group is a special no-treatment
group where treatment will be given later
Merikle & Skanes (1992)
• Experimental group: given a commercially
available subliminal tape to help lose weight
• Placebo group: given a commercially available
subliminal tape to reduce dental anxiety
• Wait-list control: told that the maximum
number of participants are currently enrolled and
they will have to be placed on a wait list
• First two groups told to listen for 1-3 hours/day,
all 3 groups were weighed weekly for 5 weeks.
Yoked control groups
• If individual participants in your treatment group
have a different experience in the study, it may
be important to equate those experiences
• Each participant in a yoked control group has
their experiences matched to a participant in the
treatment group in every way except they receive
no treatment
• e.g. Dunn et al (1996). Measuring effectiveness of eye
movement desensitization and reprocessing (EMDR) in
non-clinical anxiety: a multi-subject, yoked-control design.
Dunn et al (1996)
“Twenty-eight subjects from a university’s subject
pool were paired on sex, age, severity, and type
of stressful or traumatic incident. One subject in
each pair was selected to receive EMDR; the
experimental partner spent the same amount of
time receiving a visual (non-movement) placebo.
Subjective units of discomfort (SUD) scores and
physiological measurements were taken prior to
and following treatment. “
• “EMDR involves having the patient engage in a
series of therapist-directed, smooth pursuit eye
movements accompanied by a focus on imagery
related to the traumatic event.”
Ex Post Facto Design
- Unmanipulated independent variable
– i.e. independent variable is a subject variable - Matching after the fact to produce equivalent
groups, but no random assignment is possible
e.g. McDonald & Flanagan (2004). Social
Perception Deficits After Traumatic Brain Injury:
Interaction Between Emotion Recognition,
Mentalizing Ability, and Social Communication
McDonald & Flanagan (2004)
“Thirty-four adults with severe traumatic brain
injuries (TBI) and 34 matched control participants
were asked to interpret videotaped conversational
exchanges”
• “Study participants were asked to judge the
speakers’ emotions, the speakers’ beliefs (firstorder theory of mind), what the speakers intended
their conversational partners to believe (secondorder theory of mind), and what they meant by
remarks that were sincere or literally untrue (i.e.,
a lie or sarcastic retort)”
• “As a group [participants in the TBI group] were
generally unimpaired in their ability to interpret the
meaning of comments that were meant to be taken
literally (i.e., sincere remarks and lies), but they
demonstrated significant impairments when required
to infer the meaning of interpersonal exchanges
between people that encompassed nonliteral (i.e.,
sarcastic) remarks. They also demonstrated
significant impairments in the ability to recognize the
emotional and mental state of others.”
Repeated measures design
- Independent variable always manipulated
- Levels are made equivalent by counterbalacing
e.g. Steele, Ball & Runk (1997). Listening to
Mozart does not enhance backwards digit span
performance.
Steele, Ball & Runk (1997)
• “In a within-subjects design 36 undergraduates
were exposed to 10-min. periods of Mozart
music, a recording of rain, or silence. After each
stimulus period, undergraduates had three
attempts to hear and recall different 9-digit
strings in reverse order.”
• “The order of stimulus conditions was
counterbalanced across participants using a
Latin square design.”
Latin square
36 participants
• 3 conditions = 3! = 3x2x1 = six possible orders
– Therefore, we have 6 participants per order
Order 1 A B C
Order 2 C A B
Order 3 B C A
Steele, Ball & Runk (1997)
Remember the goal of counterbalancing is to
help equate the levels of your IV
• In a repeated measures design, order is the
main thing that could differ across the levels
• By counterbalancing, Steele et al made sure that
the type of music was not confounded with its
position in the experiment
“Following exposure to a stimulus condition, each
participant listened to three nine-digit sequences.
Digits were presented on the tape at the rate of
one every 2 sec. After each nine-digit sequence,
the participant attempted to repeat that sequence
in reverse order. The score recorded was the sum
of number correct across the three sequences, the
maximum score being 27.”
Bar vs line graphs
• Bar graphs are preferable when the independent
variable is categorical
• If the IV is continuous, or could be interpreted as
continuous, we should use a line graph
Reading graphs
- Examine the axes carefully
- Check the labels
- Check the scale
- Consider the context
- What story is the graph trying to tell?
- Could an alternate graph tell a different story?
- Is there any information missing?
