Lecture 6: Experimental Research

Question 1

What is experimental research?

Answer 1

A research methodology where the researcher seeks to observe how changes in one or more variables impact on other variables.

It allows investigation of causal relationships.

Requires high levels of control.

Question 2

What is an independent variable (IV)?

Answer 2

The variable that is manipulated (the assumed ‘cause’)

Question 3

What is the dependent variable (DV)?

Answer 3

The outcome variable that is measured (the assumed ‘effect’)

Question 4

What are extraneous variables?

Answer 4

Any other variables that impact on the relationship (e.g. ones that could weaken or distort results)

Question 5

What are confounding variables (or confounds)?

Answer 5

A type of extraneous variable that systematically varies with the independent variable (i.e. one which might otherwise be responsible for the changes in the dependent variable)

Question 6

What are controls?

Answer 6

When researchers aim to control or eliminate extraneous variables in the research design

Question 7

What are some design issues?

Answer 7

Randomisation

• random assignment of participants and treatments to groups
• scatters extraneous variables between conditions

Matching
* assigning participants to conditions based on pertinent attributes

Question 8

What are laboratory studies?

Answer 8

Experiment conducted in an artificial setting (such as a testing laboratory)

Researcher has much greater control over extraneous variables

Question 9

What are field studies?

Answer 9

Experiment conducted in a natural setting

• difficult to control extraneous variables
• natural vs. formed conditions

Question 10

What is a between-subjects design?

Answer 10

When comparisons are made between different groups of people

Question 11

What is a within-subjects design?

Answer 11

When comparisons are made using the same group of people (e.g. with pre- and post- designs)

Question 12

What is internal validity?

Answer 12

Whether variation in the DV can be confidently attributed to the variation in the IV

Question 13

What is external validity?

Answer 13

The extent to which the results of can generalise beyond the current study.

• Is it likely that this group will also be found in non-experimental settings, with other groups, etc?
• Wider population?

Question 14

What are the threats to internal validity? (7)

Answer 14

• Maturation effects
  
  • Systematic changes in the participant across time points or within the course of a study
  • Long-term maturation?
  • Short-term maturation?
• History effects
  
  • Where the participant’s environment has changed across studies or within the course of a study
  • For example, brand crisis midway through longitudinal study
• Testing
  
  • When an initial measure (e.g., pre-test) sensitises participants to the nature of the experiment
• Instrumentation
  
  • Where changes in the measurement instrument change between test and re-test
• Selection
  
  • Where participants are allocated to conditions in some systematic way
• Mortality (a.k.a. Attrition)
  
  • Refers to participants dropping out of the study
  • Problematic if there is systematic attrition
  • Consider testing of social marketing programs on topics like safe sex, reduced speeding, quitting smoking
• Demand effects
  
  • Where aspects of the experiment that provide cues to participants about researcher’s purpose, which are then acted upon by participants
  • Hawthorne effect - unintended effects cause by participants behaving differently

Question 15

What are maturation effects?

Answer 15

A threat to internal validity

• Systematic changes in the participant across time points or within the course of a study
• Long-term maturation?
• Short-term maturation?

Question 16

What are history effects?

Answer 16

A threat to internal validity

Where the participant’s environment has changed across studies or within the course of a study
- For example, brand crisis midway through longitudinal study

Question 17

What is testing (regarding threats to internal validity)?

Answer 17

When an initial measure (e.g., pre-test) sensitises participants to the nature of the experiment

Question 18

What is instrumentation (regarding threats to internal validity)?

Answer 18

Where changes in the measurement instrument change between test and re-test

Question 19

What is selection (regarding threats to internal validity)?

Answer 19

Where changes in the measurement instrument change between test and re-test

Question 20

What is mortality (aka attrition)?

Answer 20

A threat to internal validity?

Refers to participants dropping out of the study

Problematic if there is systematic attrition

Consider testing of social marketing programs on topics like safe sex, reduced speeding, quitting smoking

Question 21

What are demand effects (regarding internal validity)?

Answer 21

Where aspects of the experiment that provide cues to participants about reseacher’s purpose which are then acted upon by participants

Question 22

How can internal validity be improved? (3)

Answer 22

• Control group - no treatment or filler treatment
• Blinded and double-blinded experiments
• Random assignment to conditions

Question 23

What are 3 threats to external validity?

Answer 23

• Non-representative samples - e.g. reliance on convenience samples
• Artificial stimuli - stimuli that suit the experiment, but do not represent full range of real-life stimuli. Ecological validity?
• Artificial settings - testing in laboratory setting that does not correspond sufficiently with real-life setting

Question 24

What does X mean?

Answer 24

Exposure to IV (or a level of the IV)

Question 25

What does O mean?

Answer 25

Observation or measurement of DV

Question 26

What does R mean?

Answer 26

Random assignment of units (e.g. people to conditions)

Question 27

What is the one-shot case study (Quasi-experimental designs)?

Answer 27

A single group is exposed to some IV, and then observations are made (DV) E.g. government announces an education policy and constituents’ voting intentions are assessed Difficult to determine if IV has had any impact, or if observed results can attribute to the IV

Question 28

What is the static group comparison (Quasi-experimental designs)?

Answer 28

Experimental group vs control group Two groups are exposed to different levels of an IV (e.g. IV present, IV absent) and observations are made Problems of exposure to pre-test is eliminated Problems relate to lack of group equivalence, selection bias, etc

Question 29

What is the one group, pre-test - post-test design (Quasi-experimental designs)?

Answer 29

A single group is first observed, then exposed to some IV, and then observations are made again A pre-test baseline allows for comparison of post-test Problems relate to maturation, history, etc

Question 30

What is the time series design (Quasi-experimental designs)?

Answer 30

A single group is pre-tested multiple times, exposed to some IV, and is then post-tested multiple times Reduces potential problems associated with random fluctuations in performance, and helps to demonstrate how effective the IV is in impacting the IV long-term Problems relate to history and maturation effects, testing effects, etc Can be costly

Question 31

What is the post-test only, control group design (true experimental design)?

Answer 31

Randomisation Two groups are randomly chosen, exposed to different levels of an IV (e.g. IV present, IV absent) and observations are made Random assignment eliminates some problems associated with groups not initially being similar

Question 32

What is the pre-test - post-test control group design?

Answer 32

Two groups are randomly chosen, observations are made for each group, they each receive exposure to different levels of the IV (e.g. IV present, IV absent) and observations are made again Experimental group can be compared to control group, and also back to baseline

Question 33

What is the Solomon four-group design?

Answer 33

Allows comparison of first two groups as with pre- and post-test design but addresses problems of history, maturation and testing with the provision of the two groups that do not have the pre-test Challenges relate to having sufficient resources to execute

Question 34

What are factorial designs?

Answer 34

The manipulation of two or more IVs Sometimes effect differs across levels of one IV, when other IVs are taken into account Other factors are also likely involves, that we would be interested in

Question 35

The effect of one variable is known as what?

Answer 35

The main effect (e.g. main effect of gender or location)

Question 36

An effect due to a combination of variables is known as what?

Answer 36

An interaction effect - there is an interaction between gender and location - differences in the two levels of gender are not the same across each level of location

Question 37

What are test markets?

Answer 37

A controlled experimental procedure used by companies to trial a new product, promotion, strategy, etc. under realistic conditions with a limited audience. It's a form of field experiment.

Question 38

What are the types of test markets? (4)

Answer 38

* Standard test marketing 
- For example, company restricts distribution to limited geographic area or stores - High external validity * Control method of test marketing - For example, use of forced distribution to particular stores to ensure constantly in stock (e.g., paid shelf space) * Online test marketing - Panel used to test copy or respond to new product promotions * Virtual reality test marketing - Test market within virtual reality environment (e.g., Second Life)

Question 39

What are the limitations of test markets (4)?

Answer 39

Takes a long time to properly evaluate performance - too short can over-estimate sales (trial only purchases) Loss of secrecy to competitors Competitor tactics (e.g. distortion of results, imitation) May not yield representative results if not implemented effectively (over-zealous salespeople, regularly used test markets)