Flashcards in SCC Chapter 5 – Designs That Use Both Control Groups and Pretests Deck (20):

1

## Justification for use of pretest

###
• smaller differences on pretest = less likelihood of strong initial selection biases (though without random assignment we don’t know that confounds are unrelated to outcome

• helps with stat analyses, especially if measures are reliable

2

## Untreated Control Group Design With Pretest and Posttest Samples

###
• nonequivalent comparison group design

• most common of all quasi-experiments

3

## Design Map of Untreated Control Group Design With Pretest and Posttest Samples

###
Diagram = NR O1 X O2

NR O1 O2

4

## Selection-instrumentation threat

### difficulty measuring certain points in a scale precisely, or having some items weighted more heavily than others): more acute if groups are unequal

5

## Selection-regression

### groups selected are not equally matched due to performance (gifted children matched with non-gifted children in other group will bias results

6

## Selection-history

### an event occurred between pre- and post- treatment that biases results

7

## Outcome 1: Both Groups Grow Apart in the Same Direction: selection-maturation

### “fan-spread model” – standardizing scores makes “fan” disappear because the variance is equalized. These effects may be spurious if groups are unequal

8

## Outcome 2: No Change in the Control Group

### treatment group may be older or other maturation issues

9

## Outcome 3: Initial Pretest Difference Favoring the Treatment Group That Diminish Over Time

### superiority of tx group is diminished or eliminated at posttest

10

## Outcome 4: Initial Pretest Differences Favoring the Control Group Diminish Over Time

### this is desirable, for instance, if a school implements a tx program for underachieving students. Outcome is subject to scaling and history threats

11

## Outcome 5: Outcomes that Cross Over in the Direction of Relationships

### amenable to causal interpretation. Power to detect a statistically reliable interaction in this type of study is low. Should not rely on research design to obtain this type of result

12

## 5 Outcome Patterns that are Plausible for Different Result Scenarios

###
Outcome 1: Both Groups Grow Apart in the Same Direction: selection-maturation

Outcome 2: No Change in the Control Group

Outcome 3: Initial Pretest Difference Favoring the Treatment Group That Diminish Over Time

Outcome 4: Initial Pretest Differences Favoring the Control Group Diminish Over Time

Outcome 5: Outcomes that Cross Over in the Direction of Relationships

13

## Ways to Improve the Untreated Control Group Design With Dependent Pretest and Posttest Samples

###
1. Using a Double Pretest

2. Using Switching Replications

3. Using a Reversed-Treatment Control Group

4. Matching Through Cohort Controls

5. Matching Through Cohort Controls by Adding Pretests

6. Improving Cohort Designs With a Nonequivalent DV

7. Combining Switching Replications With a Nonequivalent Control Group Design

14

## Using a Double Pretest

### understand biases pre-treatment (from pretest1 to pretest 2). Also permits assessment of a selection-maturation threat on the assumption that the rates between the 2 pretests will continue between the second pretest and posttest. This is testable ONLY for the untreated group. It can be difficult to get a 2nd pretest together

15

## Using Switching Replications

### Researcher administers tx at a later date to the group that initially served as a no treatment control. The second phase is not an exact replication. Obvious contextual differences between the groups (one receives treatment before the other, but they both receive O2 at same time). Problem: you can’t remove tx from the 1st group at O3

16

## Using a Reversed-Treatment Control Group

### X+ intended to produce an effect in one direction, while X- is intended to produce the opposite. Assumes that little historical or motivational change is taking place.

17

## Matching Through Cohort Controls

###
Cohort = the successive groups that go through processes such as graduating from school, etc. cohorts can be useful as control groups if 1) one cohort experiences a given tx and ealier or later cohorts do no; 2) cohorts differ only in minor ways; 3) organizations insist that tx be given to everyone (making control impossible; 4) an org’s records can be used for constructing and comparing cohorts

Cohorts will never be as comparable as groups that are randomly assigned

18

## Improving Cohort Controls by Adding Pretests

### Compare cohort pretest means to assess for nonequivalence; Pretest increases statistical power by allowing use of within-subject error terms. Enables assessment of maturation and regression and enters into better statistical adjustment for group nonequivalence. History is a salient internal validity threat in this design

19

## Improving Cohort Designs With a Nonequivalent DV

### improve a study with a specific measure of DV

20