Types of Experimental Designs

Question 1

What are the four main types of experimental designs?

Answer 1

Between-subjects, within-subjects, mixed factorial, quasi-experimental

Question 2

What makes it a Between-Subjects Experimental Design?

Answer 2

• Manipulate 1+ factors (IVs) to observe their effects on 1+ DVs
• Respondents are randomly assigned to one and only one group
• At least 50 participants per group
• Groups are orthogonal (equal)

Question 3

2(+) factor between subject design let’s us look at ______

Answer 3

• main effects of each IV separately on DV
• examine the interaction between the two IV’s
• look at different combos of the IV’s on the same DV

Question 4

If there is a 2 x 3 factors, how many groups are there?

Answer 4

6 (2*3)

Question 5

a 2 x 4 has how many levels in each?

Answer 5

2 levels x 4 levels

Question 6

True or False: The more factors you have, the better

Answer 6

False: the more factors you have and the more levels within each factor, the more complex analyses get

Question 7

What is a 2 x 2 x 2 design?

Answer 7

3 factor design

Question 8

Advantages Between-Subjects Designs

Answer 8

• simple to execute
• more conservative than within-subject designs
• respondents are less likely to suffer from survey fatigue

Question 9

Disadvantages Between-Subjects Designs

Answer 9

• respondents give their answer in a “vacuum” with no natural anchor/comparison points
• less power than within-subjects (only 1 data point per person)
• potentially harder to get significant effects b/c differences in stimuli may be less impactful when they are presented independently

Question 10

Between-subjects allows for judgments in ___-

Answer 10

isolation

Question 11

Within-subjects allows for judgments _____

Answer 11

relative, contextual comparisons among multiple alternatives

Question 12

What does it mean to have repeated measures?

Answer 12

Participants are assigned to more than 1 level of the same factor/IV (i.e. more than 1 group)

Question 13

How is causality inferred in within-subject designs?

Answer 13

Causality is inferred by changes in the repeated measure(s)

Question 14

One factor vs. two factor within-subject designs

Answer 14

one factor only asks 1 repeated measures, two factor has two different factors involved

Question 15

True or False: within-subject designs are good for pretests?

Answer 15

True

Question 16

Advantages of Within-Subject Designs

Answer 16

• gives you more power with less respondents
• better controls for error variance than between-subjects (exact same people are answering questions)
• allows for direct, relative comparisons between 2 or more stimuli (more consistent with real world)
• encourages respondents to use the provided stimuli as anchors/basis for comparisons (rather than something random we can’t control)

Question 17

Disadvantages of Within-Subject Designs

Answer 17

Carryover and demand effects - participation in one task/stimuli may affect participation in another
- practice: respondents perform better in 2nd task because of familiarity
- fatigue: respondents perform worse in 2nd task because of survey fatigue

respondents may heel compelled to give different answers to different stimuli

less conservative than between-subjects

Question 18

Two things to remember to do/is good to do in within-subject designs?

Answer 18

Randomize the order that the levels of the repeated measures are shown and asked about

put in a “filler” task between the repeated levels

Question 19

Mixed Factorial Design

Answer 19

Uses both between-subject and within-subject designs in one overall design.

At least one factor is between-subject (i.e., respondents are randomly assigned to only 1 level of this factor) and at least one factor is within-subject (i.e., everyone sees all levels of this factor)

Question 20

What type of design:
Respondents look at 9 bags of chips on a retail shelf (3 are pre-determined by researchers to be healthy, 3 are moderately healthy, and 3 are unhealthy). Each respondent indicates his/her PI for 1 healthy, 1 moderately healthy, and 1 unhealthy item that are prechosen by researchers.

Answer 20

Within-Subjects Design

Question 21

What type of design:
Respondents look at 9 bags of chips on a retail shelf (3 are pre-determined by researchers to be healthy, 3 are moderately healthy, and 3 are unhealthy). Each respondent indicates his/her PI for 1 healthy, 1 moderately healthy, and 1 unhealthy item that are prechosen by researchers. We randomly assign half of the respondents to be exposed to a “organic” health claim on the front of the packages and the other half of respondents don’t see any claims at all (i.e., a control condition).

Answer 21

Mixed Factorial Design

Question 22

Quasi Experimental Design Defining Features

Answer 22

Takes place in field setting where we observe a DV before and after a change in IV (which occurs either naturally or from a manipulation)

Lack of random assignment.

Question 23

Example of why random assignment not always feasible/practical or ethical?

Answer 23

field studies; when using secondary data
studying child abuse

Question 24

Where does internal validity fall in quasi-experiments?

Answer 24

falls somewhere b/w correlational studies and true experiments.

Question 25

Is it difficult to infer causality with quasi-experiments?

Answer 25

Yes, results should be interpreted with caution

Question 26

Advantages of Quasi-Experimental Designs

Answer 26

- High external validity (generalizability) - keeps respondents from being aware of the treatments/manipulations in other groups

Question 27

Disadvantages of Quasi-Experimental Designs

Answer 27

- lacks internal validity - causal inferences/conclusions should be drawn since we cannot rule out alternative explanations

Question 28

Non-Equivalent Groups Design

Answer 28

Researchers choose 2 or more existing groups in the “real world” that appear as similar as possible (ex: two similar communities, schools, stores, consumer groups). But since people aren’t randomly assigned to the groups, the groups likely actually differ in some ways (i.e., are non-equivalent). It’s a between-subject design w/ no random assignment

Question 29

What type of design: We want to know if grades are higher in online classes vs. in-person classes. Students sign up for 1 of 2 marketing principles sections (not knowing what the format will be). Then, 1 section is taught online and the other is taught in person. At the end of the semester, we compare the average GPA’s of each section.

Answer 29

Non-Equivalent Groups Design

Question 30

Pretest - Posttest Design

Answer 30

The same DV is measured among the same group of people (one group) before (i.e., the pretest) and after (i.e., the posttest) a treatment/manipulation is administered. Then we compare the averaged DV before and after the manipulation (i.e., looking for any change in it). Main problem is that we don’t have another similar group (i.e., a control group) to compare this treatment group to (like we do with a non-equivalent groups design).

Question 31

What type of design: You first measure 100 donors’ attitudes toward Ole Miss. Then they all attend a huge banquet in their honor on the Ole Miss campus where they are publically recognized. After, you measure those same 100 donors’ attitudes toward Ole Miss (using the same 1 to 7 scale). Then you compare average attitudes before and after.

Answer 31

Pretest-posttest design

Question 32

Interrupted Time Series Design (compared to pretest, posttest)

Answer 32

A variant of the pretest-posttest design. Similar to pretest-posttest in that we measure a DV both before and after a naturally occurring treatment (e.g., new smoking ban on campus; new mandatory food label). Different from pretest-posttest in that it includes multiple, equally-spaced pretest and posttest DV measurements (e.g., daily, weekly, monthly, yearly).

Question 33

How is pretest-posttest similar to a within-subject design

Answer 33

each person first gives a response under a control condition (pretest) and then under a treatment condition (posttest).

Question 34

How is pretest-posttest different to a within-subject design

Answer 34

the order of the conditions aren’t counterbalanced (i.e., everyone gets the control condition first, then the treatment condition).

Question 35

Interrupted Time Series Design (defining measures/qualities)

Answer 35

compares the trend in our DV over time before treatment to the trend in that same DV over time after the treatment. The intervention time frame needs to be clearly defined and not included in either the pre or post periods. Arguably the best quasi-experimental design in terms of causal inference!

Question 36

What type of design: The city of Oxford increases the sales tax rate to 12% on January 1. You want to know if/how that affected sales of a retail store in Oxford. You record the overall sales in dollars of a store in Oxford for each month for the immediate 12 months leading up to the sales tax hike (so 12 measurements). Then, you record the overall sales in dollars of the store for each of the immediate 12 months following the tax hike (so 12 more measurements). Lastly, you compare the sales before and after the tax hike.

Answer 36

Interrupted Times Series Designs

Question 37

What type of design: You want to know if/how hiring a new manager affected sales in a retail store. You record the overall sales in dollars of a store each Sunday for two months immediately leading up to the new hire (so 8 measurements). Then, you record the overall sales each Sunday for the two months immediately after the new hire (so 8 more measurements). Lastly, you compare the sales before and after the new hire.

Answer 37

Basic Time Series Design

Question 38

Types of Quasi-Experimental Designs

Answer 38

Non-Equivalent Groups Design Pretest – Posttest Design Basic and Interrupted Time Series Designs

Question 39

When do you have a cross-sectional study?

Answer 39

Regardless of design, if you don’t manipulate ANY IV’s– and instead use ONLY measured IV’s – then you aren’t running an experiment. You have a cross-sectional study!

Question 40

When can you NOT infer causality?

Answer 40

When there is not at least 1 manipulated variable (and random assignment wasn't possible)

Question 41

What type of design: Respondents were presented with 3 different nutrition package icons (one at a time). They answered the same set of questions about each (e.g., how specific each icon is, how helpful each icon is, etc.).

Answer 41

One factor within-subjects design

Question 42

What type of design: You have a 2 (device type: phone vs. tablet) x 2 (shopping list: short vs. long) within-subjects design. Each respondent is tasked with finding products on a shopping list and adding them to their basket on a certain website. Each respondent fulfills both the short and long list and does so on both a phone and a tablet.

Answer 42

Two factor within subjects design

Question 43

What type of design: You have a 2 (hunger level: more hungry vs. less hungry) x 3 (healthfulness level: healthy vs. moderate vs. unhealthy) within-subjects design. Each respondent indicates his/her PI for the healthy, moderate, and unhealthy items when more hungry. Then each respondent indicates his/her PI for each item when he/she is less hungry.

Answer 43

Two factor within subjects design

Question 44

What type of design: Every respondent looks at a group of 9 bags of chips on a retail shelf (3 are pre-determined by researchers to be healthy, 3 are moderately healthy, and 3 are unhealthy). The only difference between the chips are the healthfulness of them. Each respondent then indicates his/her purchase intent for 1 healthy, 1 moderately healthy, and 1 unhealthy item prechosen by researchers (on the same 1 to 7 scale).

Answer 44

One factor within subjects design

Question 45

What type of design: Every respondent gives their attitude toward 4 different potential packages for a new brand of cereal that differ only on color

Answer 45

One factor within subjects design

Question 47

What are the four main types of experimental designs?

Answer 47

Between-Subject, Within-Subject, Mixed Factorial, Quasi-Experimental

Question 48

What is the most popular experimental design type?

Answer 48

Between-Subject Design

Question 49

What is a Between-Subjects Experimental Design?

Answer 49

A design where one or more IVs are manipulated, and respondents are randomly assigned to only one group/condition.

Question 50

Why is random assignment important in Between-Subjects design?

Answer 50

It ensures that differences between groups are due only to the manipulation.

Question 51

What is the minimum recommended number of respondents per group in a Between-Subjects design?

Answer 51

At least 50, but more is better for statistical power.

Question 52

What is an example of a 1-factor Between-Subjects design?

Answer 52

Testing whether a funny or sad ad influences donation intentions.

Question 53

What is an example of a 2-factor Between-Subjects design?

Answer 53

Testing ad type (funny vs. sad) and ad length (short vs. long) on donation intentions.

Question 54

What does a 2x2 Between-Subjects design allow researchers to examine?

Answer 54

The main effects of each IV separately and the interaction between them.

Question 55

What is the purpose of a control group in a Between-Subjects design?

Answer 55

To compare against manipulated conditions to measure true effects.

Question 56

What are the advantages of a Between-Subjects design?

Answer 56

Simple to execute, avoids carryover effects, and reduces survey fatigue.

Question 57

What are the disadvantages of a Between-Subjects design?

Answer 57

Requires more respondents, responses are given in isolation, and can have less statistical power.

Question 58

What is a Within-Subjects Experimental Design?

Answer 58

A design where participants experience more than one level of the IV, allowing for relative comparisons.

Question 59

What is another name for a Within-Subjects design?

Answer 59

Repeated Measures Design.

Question 60

What is an advantage of Within-Subjects designs over Between-Subjects?

Answer 60

More power with fewer respondents since each respondent gives multiple answers.

Question 61

What is an example of a 1-factor Within-Subjects design?

Answer 61

Respondents rate the healthfulness of multiple food products.

Question 62

What is an example of a 2-factor Within-Subjects design?

Answer 62

Measuring purchase intent for different food healthfulness levels when hungry vs. not hungry.

Question 63

What are disadvantages of Within-Subjects designs?

Answer 63

Carryover and demand effects, potential fatigue, and respondents may give different answers based on context.

Question 64

How can researchers minimize carryover effects in Within-Subjects designs?

Answer 64

Randomize the order of conditions and use filler tasks between conditions.

Question 65

What is a Mixed Factorial Experimental Design?

Answer 65

A combination of Between-Subjects and Within-Subjects design in one study.

Question 66

What is an example of a Mixed Factorial design?

Answer 66

Testing how health claims (present vs. absent) and product nutrition level (healthy, moderate, unhealthy) affect purchase intent.

Question 67

What is a Quasi-Experimental Design?

Answer 67

A study that lacks random assignment and often occurs in field settings.

Question 68

Why can’t causality be inferred from Quasi-Experiments?

Answer 68

Because of the lack of random assignment, alternative explanations can't be ruled out.

Question 69

What are common Quasi-Experimental designs?

Answer 69

Non-Equivalent Groups Design, Pretest-Posttest Design, and Interrupted Time Series Design.

Question 70

What is a Non-Equivalent Groups Design?

Answer 70

Comparing pre-existing groups (e.g., comparing student performance in online vs. in-person classes).

Question 71

What is a Pretest-Posttest Design?

Answer 71

Measuring a DV before and after an intervention within the same group.

Question 72

What is an Interrupted Time Series Design?

Answer 72

Measuring a DV at multiple time points before and after an intervention (e.g., tracking sales before and after a tax increase).

Question 73

What is a Cross-Sectional Study?

Answer 73

A study that only measures data at one point in time without manipulating variables.

Question 74

Why can’t causality be inferred from Cross-Sectional Studies?

Answer 74

Because they lack manipulated variables and random assignment.