Lecture 21: Non-Experimental and Quasi-Experimental Designs Flashcards
1
Q
non- and quasi-experimental designs
A
- In both non- and quasi-experimental designs, there is no manipulation (by the experimenter) of the IV
- Researchers compare pre-existing groups on a DV of interest
- Groups are defined by time or participant characteristics
- Both non-and quasi-designs produce groups of scores to be compared for significant differences
2
Q
characteristics of non- and quasi-experimental designs
A
- No random assignment
- We cannot determine causality
- Groups can differ on other factors: internal validity issues
3
Q
research design terms used in quasi- and non-experimental designs
A
- Quasi-independent variable (IV): the variable used to differentiate the groups of participants or the groups of scores being compared
- Dependent variable (DV): the variable that is measured
4
Q
similarities between non- and quasi-experimental designs
A
- No manipulation of the IV
- No random assignment
5
Q
differences between non- and quasi-experimental designs
A
Quasi-experimental uses controls for confounds to keep internal validity high, but non-experimental doesn’t
6
Q
IV in non- and quasi-experimental designs
A
- Typically, the IV is a participant variable (age, marital status) or a time variable (pre- or post-event)
- The researcher usually cannot control the assignment of participants to groups because the groups already exist
7
Q
3 general categories of non- and quasi-experimental research
A
- between-groups design
- within-groups design
- developmental research design
8
Q
types of between-groups designs
A
- Differential design: non-experimental
- Post-test only NEG design: non-experimental
- Pre-test/post-test NEG design: quasi-experimental
9
Q
types of within-groups designs
A
- One group pretest-posttest design: non-experimental
- Time series design: quasi-experimental
10
Q
types of developmental research designs
A
- Cross-sectional designs: between-subjects 2 or more groups
- Longitudinal designs: within-subjects 1 group, measured 2 or more times
- Cross-sectional and longitudinal: between- and within-subjects 2+ groups, measured 2+ times
11
Q
non-equivalent group (NEG) between-groups design
A
- Pre-existing groups are used
- Groups are differentiated by one specific factor
- The purpose is to show that this factor is responsible for differences between the scores of the groups
- Called “non-equivalent groups” because the experimenter cannot control the group’s membership (or individual differences)
12
Q
assignment bias in non-equivalent group (NEG) between-groups design
A
- These designs raise the problem of assignment bias
- Groups may differ on a variable other than the IV and this can be a possible confound
- Precludes cause-and-effect explanations
- Because of this lack of control, there is a threat to internal validity
13
Q
differential designs (non-experimental)
A
- Also called an ex-post facto (after the fact) design
- Simplest of the between-group (NEG) designs
- Individual differences are the primary interest: form groups based on these differences
- Simply compare these pre-existing groups to find a difference in the variable of interest
- No manipulation of control of assignment of participants to groups; just comparing them on pre-existing characteristics
14
Q
example of a differential design
A
SES, stress, and cognitive function study found that there were no differences in cognitive functioning between high and low SES groups but there were early stress differences in the younger groups
15
Q
Post-test-only NEG designs
A
- Interested in examining the effectiveness of a treatment or intervention (applied research)
- No manipulation or control over who is exposed to the treatment or intervention
- Compare those who received treatment vs. those who did not, based on pre-existing groups
- No control over the assignment of participants to groups; one group is measured after tx, other group is measured at the same time (but received no treatment)
- Commonly used when tx is given to a cluster of individuals and compared to another cluster
16
Q
assignment bias in post-test-only NEG designs
A
This method does not address the problem of assignment bias
17
Q
pre-test/post-test NEG designs
A
- With this design, we try to add a bit of control by including a pre-test measurement for both groups
- Much stronger version than the NECG
- Limits threats to internal validity
- Can compare groups before treatment and look at differences pre- and post-treatment
18
Q
how can you reduce threats to internal validity in pre-test post-test NEG designs
A
- Determine how similar the two groups are, before administering treatment by comparing pre-treatment measures
- If similar at pre-test, there is evidence that Ps in one group is not substantially different from those in another group
- Although this does not make the groups equivalent, it does reduce the impact of assignment bias on the results
- With this technique, you are controlling for one specific variable
- With randomization in a true experimental design, you can target multiple variables
19
Q
what threats are pre-test post-test NEG designs vulnerable to?
A
the same threats to internal validity as experimental designs (history, instrumentation, testing effects, maturation, and statistical regression)
20
Q
how should we interpret the following findings in a pre-test post-test design?: participants are similar at pre-test but there is a post-treatment difference between groups
A
we can be somewhat confident that it is due to the treatment
21
Q
advantage of pre-test post-test NEG designs
A
limit time-related threats
22
Q
one-group non-experimental pretest-posttest designs
A
- Studies in which a series of observations are made over time in just one group of participants
- There is no attempt to control threats to internal validity
- Two observations are made for each participant, one before and one after treatment
23
Q
goal of one-group non-experimental pretest-posttest designs
A
evaluate an intervention/treatment by comparing observations made before with those made after
24
Q
assignment bias in one-group non-experimental pretest-posttest designs
A
- We are not comparing groups (no control group) but rather individual scores before and after treatment
- The problem of assignment bias is eliminated
25
downsides of one-group non-experimental pretest-posttest design
- You cannot counterbalance the order of treatment as you can in experimental designs
- You can't be sure that a change post-treatment is due to the treatment and not some other factor
26
threats to one-group non-experimental pretest-posttest designs
Threatened by time-related factors (history, instrumentation, testing effects, maturation, regression)
27
time-series designs
- Attempt to add some control by making a series of observations for each participant
- Some observations before and some after treatment
- Similar to single-case designs
28
Value of repeated pretest and posttest observations within participants:
- Allows researchers to see existing trends in data
- Identify practice, fatigue, maturation, instrumentation, and regression, since they should be measurable before treatment
- Allows researchers to observe post-treatment trends
- Can examine manipulated treatments
- Can look at non-manipulated events (data from archives) and societal changes
29
how many data points are needed in a time-series design?
- You need at least 3 data points before the treatment and 3 after
- Multiple observations allow you to see if the change is due to your treatment or not and examine post-treatment trends to see if the effect of X fades
30
threats to internal validity in time-series designs
- An external event can be a threat to internal validity in time series designs but only if it occurs at the same time as the treatment
- If treatment and outside events occur at different times, one can separate the effects in the time series using statistical techniques
31
cross-sectional developmental research designs
- A between-groups design that uses different groups for each age or age bracket
- The outcome variable is measured for each group at one point in time and the groups are compared
- When a study evaluates differences related to age, the design is typically called cross-sectional
32
example of a cross-sectional developmental research design
differences in ease of using technology by age group
33
advantages of cross-sectional developmental research designs
- Time-efficient (one measurement time-point)
- No long-term cooperation required
34
disadvantages of cross-sectional developmental research designs
- Cohort or generation effects (changes due to era or period)
- Individual changes are not assessed
35
longitudinal developmental research designs
- A within-groups design that measures a variable in the same group of individuals over time
- Participants have usually grown up in similar circumstances
- One group’s pretest-posttest measures of health are collected
- The treatment is age
36
advantages of longitudinal developmental research designs
- No cohort or generation effects
- Individual changes are assessed
37
disadvantages of longitudinal developmental research designs
- Time-consuming and costly
- Participant attrition can cause unbalanced measures
- Potential for practice effects over time (repeated tasks)
38
cross-sectional longitudinal designs
- Also known as mixed developmental designs
- Compare results that were obtained from separate samples (cross-sectional) that were obtained several times (longitudinal)
- Allows for the examination of the development of behaviours due to mechanisms other than individual aging
- Can draw comparisons within groups (across time) and between groups (same time point)
39
example of a cross-sectional longitudinal design
influence of sport on physical development
40
Major distinction of non- and quasi-experimental designs from experimental designs
no random assignment of participants to conditions
41
Major distinction of non-experimental from quasi-experimental designs
no attempt at control comparison in non-experimental designs
