research methods exam 3

Question 1

3 criteria for a causal claim

Answer 1

-temporal precedence
-covariance
-no extraneous variables

Question 2

construct validity

Answer 2

-how well a conceptual variable is
operationalized/defined
-When you ask how well a study measured or manipulated a
variable, you are interrogating the construct validity

Question 3

effect size (r)

Answer 3

the strength of a relationship between two or more variables
-indicates the importance of a relationship (weak, moderate, strong)

Question 4

confidence intervals

Answer 4

precision of results
-results of statistically significant when the CI doesn’t contain zero
-given range indicated by a lower and upper value that is designed to capture the population value for some point estimate (e.g., percentage, difference, or correlation)
- a high proportion of CIs will capture the true
population value
-more people shrinks the CI, less people makes CI smaller
-if not given p value in results, use CI: significant results will not contain 0

Question 5

third variable problem

Answer 5

it is possible for a third variable (confounding/extraneous) to be causing spurious correlations
-we shouldn’t underestimate this possibility of these types of unmeasured variables

Question 6

independent variable

Answer 6

the manipulated variable
-assigning participants to be at one level or the other

Question 7

dependent variable

Answer 7

the measured variable
-aka the outcome variable

Question 8

control variable

Answer 8

variable that experimenter holds constant on purpose

Question 9

counterbalancing

Answer 9

-a way to avoid order effects
-presenting the levels of the IV to participants in different sequences
-this should cause order effects to cancel each other out when all of the data is combined
-counterbalancing can be “full” or “partial”
-full: all possible condition orders are represented
-partial: some, but not all, of the possible condition orders are represented

Question 10

manipulation check

Answer 10

an extra dependent variable that researchers can insert into an experiment to convince them that their experimental manipulation worked

Question 11

within-subjects design

Answer 11

an experimental design in which each participant is presented with all levels of the independent variable

Question 12

between-subjects design

Answer 12

-aka independent-groups design
-experimental design in which different groups of participants are exposed to different levels of the IV, such that each participant experiences only one level of the IV

Question 13

matched design/group

Answer 13

an experimental design technique in which participants who are similar on some measured variable are grouped into sets
-the members of each matched set are then randomly assigned to different experimental conditions

Question 14

pre-post design

Answer 14

experiment using an independent-groups design (between-subjects) in which participants are tested on the key dependent variable twice: once before and once after exposure to the IV

Question 15

repeated measures design

Answer 15

an experiment using a w/i groups design in which participants respond to a dependent variable more than once, after exposure to each level of the IV

Question 16

post-only design

Answer 16

experiment using an independent-groups design (between-subjects) in which participants are tested on the DV only once

Question 17

concurrent measures design

Answer 17

an experiment using a w/i-groups design in which participants are exposed to all the levels of an IV at roughly the same time, and a single attitudinal or behavioral preference is the dependent variable

Question 18

order effects

Answer 18

one of 12 threats to internal validity
-in a w/i-groups design, threat where exposure to one condition changes participants responses to a later condition

Question 19

selection effects

Answer 19

one of 12 threats to internal validity
-when the kinds of participants in one level of the IV are systematically different from those in the other
-can happen if participants choose which group they want to be in OR if researchers assign one type of person to one condition (ex:women) and another type of person to another condition (ex:men)

Question 20

design confounds

Answer 20

one of 12 threats to internal validity
-experimenter’s mistake in designing the IV
-occurs when a second variable happens to vary systematically
along with the intended independent variable
-the accidental second variable
is therefore an alternative explanation for the results

Question 21

factorial notation/design

Answer 21

-a way for researchers to test for interactions
-design is one in which there are two or more IVs (also referred to as factors)
-most common factorial design:
researchers cross the two independent variables; study each possible combination of the independent variables

Question 22

main effects

Answer 22

in factorial designs, the overall effect of one IV on the DV, averaging over the levels of the other IV
-the number of IVs = number of main effects

Question 23

interaction effects

Answer 23

-result from a factorial design, in which the difference in the levels of
one independent variable changes, depending on the level of the other
independent variable
-a difference in differences
-aka an interaction

Question 24

within, between and mixed designs

Answer 24

between-groups factorial design:
-both IVs are studied as independent-groups
-if the design is a 2×2, there are four different groups of participants in the experiment
within-groups (or repeated-measures) factorial design:
-both independent variables are manipulated as within-groups
-If the design is 2×2, there is only 1 group of participants but they participate in all 4 combinations of the design
mixed factorial design:
-one independent variable is manipulated as
independent-groups and the other is manipulated as within-groups

Question 25

interaction graphs

Answer 25

in interaction graphs where the variables DO interact, there will be lines crossing
-interaction graphs can look different, but if there is interaction, then the lines will be crossing at some point
-no interaction = no crossover

Question 26

history threat

Answer 26

one of 12 threats to internal validity
-a “historical” or external factor that systematically affects most members of the treatment group at the same time as the treatment itself
-makes it unclear whether the change is caused by the treatment received
-external factor must affect most people in the group in the same direction (systematically), not just a few people (unsystematically)
ex: why did the dorm residents use less electricity?
Was it the Go Green campaign? Perhaps. But a plausible alternative
explanation is that the weather got cooler and most residents did not use air conditioning as much
-to fix this: use comparison groups

Question 27

attrition threat

Answer 27

one of 12 threats to internal validity
-when only a certain kind of participant drops out of the study before it ends
-a reduction in number of participants from pretest to posttest
to fix this: look at the dropout levels (from which groups?) to see if the study groups remain equal

Question 28

maturation threat

Answer 28

one of 12 threats to internal validity
-change in behavior of the experimental group that occurs spontaneously over time
-to fix this: use comparison groups

Question 29

instrumentation threat

Answer 29

one of 12 threats to internal validity
-when a measuring tool changes over time
-makes it unclear if change is from different test or actual change
-to fix: mirror pre/post, use post-only and have a clear coding guide

Question 30

regression threat

Answer 30

one of 12 threats to internal validity
-extremely low or high performance at time 1 is likely to be less extreme at time 2 (closer to the average)
-happens only when measuring twice & with extreme scores
-to fix: use comparison groups

Question 31

testing threat

Answer 31

one of 12 threats to internal validity
-type of order effect in which participants’ performance improves over time because they become practiced at the dependent measure
-aka fatigue effect or practice effect

Question 32

observer bias

Answer 32

one of 12 threats to internal validity
-bias caused by researchers expectations influencing how they interpret the results
-to fix: use a double-blind study

Question 33

demand characteristics

Answer 33

one of 12 threats to internal validity
-a cue that leads participants to guess a study’s hypothesis or goals
-participants change their behavior in the expected direction
to fix: use a double-blind study, masked as best as possible

Question 34

placebo effects

Answer 34

one of 12 threats to internal validity
-people receive a treatment and improve, but only because they believe they are receiving a valid and effective treatment
to fix: add a 3rd group that receives nothing (not even placebo pill)

Question 35

when to use each type of test (t-test, ANOVA, etc.)

Answer 35

independent-samples T test:
-comparing b/w subjects, 2 different groups
T test:
-analyzing simple experiments
repeated-measures T test (aka paired samples T test):
-comparing w/i subjects (participants see both conditions), 2 means/groups
-using pre-post test
1-way ANOVA:
-3+ groups
Bivariate correlation
-2 measured, ordinal, or ratio variables
Pearson correlation
-finding relation between quantitative variables

Question 36

possible reasons for not enough between-groups difference

Answer 36

-floor and ceiling effects (floor: all scores clustered at low end, ceiling: all scores clustered at high end)
-weak manipulation of variables (ex: difference in violence after 1 hr or 2 hrs on video games - not enough difference)
-insensitive instruments (researchers have not used an operationalization of the dependent variable with enough sensitivity; ex: If medication reduces fever by 1/10th degree, you wouldn’t be able to detect it with a thermometer that was calibrated in 1 degree increments)

Question 37

bar graphs

Answer 37

comparing 2 means
-used for independent samples T test
-when you have 1 categorical and 1 quantitative variable

Question 38

scatter plot

Answer 38

-when you have 2 quantitative variables