Week 2 (Advances Issues in Experimental Research Methods)

Question 1

What is a hypothesis

Answer 1

-A specific, testable claim or prediction about what you expect to observe given a set of circumstances
-Tentative statement about the assumed relationship between two (or more) variables

Question 2

Research Hypothesis (H1)

Answer 2

-The statement you’re testing
-What you expect to find

-Directional (one-tailed): “People who do yoga will have higher wellbeing scores than those who don’t do yoga
-Nondirectional (two-tailed): “There will be a difference in wellbeing between people who do yoga and people who do not

Question 3

Null Hypotheses (H0)

Answer 3

-Provides a baseline against which to evaluate our alternate hypothesis
-It states an effect is absent

Question 4

What do surveys and experiments test for

Answer 4

Surveys
-Allow test for correlation

Experiment
-Allow test for causation

Question 5

Interpreting Correlation

Answer 5

-Correlation means that two variables vary together - as one variable changes, the other variable changes as well

Question 6

Spurious Correlation

Answer 6

When variables are correlated but not casually related

Question 7

Causation

Answer 7

Implication is:
-Change in A is associated with change in B
-Change in A reliably precedes change in B
-Without change in A, change in B does not occur

Question 8

Extraneous vs Confounding variables

Answer 8

Extraneous variable
-Anything other than the independent variable that could affect the dependent variable

Confounding variable
-A type of extraneous variable that not only affects the dependent variable, but also varies with the independent variable in a systematic manner
-It is uncontrolled and obscure the causal effect sought
-It seriously limits researcher’s claim that there is a causal link between IV and DV

Question 9

Between subjects design

Answer 9

-Allocate people to different conditions
-Each participant is tested in only one condition
-Look at difference in performance between groups

Question 10

Within subjects design

Answer 10

-Each participant is tested in all conditions
-Look at differences in performance between levels of test

Question 11

Pros and Cons of between-subjects design

Answer 11

Advantages
-Useful when impossible for an individual to participate in all conditions
-No carry-over effects

Disadvantages
-Noise from random individual differences between groups –> Less statistical sensitivity
-Greater expense (more participants, time, money)

Question 12

Pros and Cons of within-subjects design

Answer 12

Advantages
-Economical (less participants, time, money)
-Less noise (from differences between groups) –> better statistical sensitivity

Disadvantages
-Need strategies to avoid carry-over effects
-Not suitable for cases where participant must be naive for each condition

Question 13

How do we know what a good number of samples is?

Answer 13

-No easy answer: A lot to do with statistical power and effect size related to study
-Generally (but not always), the more, the better

Question 14

Effect size

Answer 14

-A statistical measure of the magnitude of an observed effect in a population
*How big the difference between two experimental groups is
*How strong a correlation is

Question 15

Power and Type II errors

Answer 15

Type II error: Failing to detect an effect that actually exists
Statistical Power is the probability of detecting a true effect when it actually exists in your population

Power = 1 -β

β = probability of making Type II error

Typically acceptable power: 80% (β = 20%)

Question 16

Validity

Answer 16

Whether the test/questionnaire measured what it intended to measure (valid tool)

Question 17

Internal validity

Answer 17

-The extent to which we can be sure that the changes we observe have actually been caused by our manipulation, rather than other factors
-In other words, how confident we are that the cause-and-effect relationship cannot be explained by other factors.
-A measure of how well our experiment was designed and executed

Question 18

Internal validity terminology

Answer 18

Pre-test (O1) = The observation or measure before the intervention
Experimental treatment (X) = The different intervention or conditions
Post-test (O2) = The observation or measure after the intervention

Question 19

Threats to internal validity

Answer 19

The key condition for internal validity
-No confounding factors can explain the DV

Question 20

Some threats to validity in between subjects design

Answer 20

-Maturation effects
-History effects
-Testing effects

Question 21

Maturation effects:

Answer 21

Participants behaviour changes over time naturally (i.e, nothing to do with treatment/investigation)

Question 22

History effects

Answer 22

Something changes about the participants circumstances that influences the variables (e.g. good/bad life events, cultural events etc.)

Question 23

Testing effects

Answer 23

Merely having been tested before may have changed how they do on the post-test

Question 24

Threats to internal validity

Answer 24

-Regression towards the mean
-Initial non-equivalence of groups
-Differential attrition

Question 25

Regression towards the mean

Answer 25

An extreme score is likely to become more average -Applied research attributing improvement to intervention -Selection effects (e.g, participants are selected because of their extremity on the variable of interest)

Question 26

Importance of control group

Answer 26

-Control groups help minimise any influence of maturation effects/ testing effects/ history effects on the conclusions we reach -Control groups don't remove these effects -But we can account for potential influence of them, and see effect despite them.

Question 27

Types of control group

Answer 27

-Active -Passive -Wait list

Question 28

Active control group

Answer 28

-Participants do something that they reasonably assume might have an effect but the researchers assume don't (sham meditation, non-effective training , placebo)

Question 29

Passive control group

Answer 29

-Participants do nothing/ a meaningless alternative task (e.g, search the internet, complete a crossword). Also called the no-treatment control group.

Question 30

Wait-list control group

Answer 30

Participants are waiting to take part in the intervention/ experimental conditions, and believe they will be at some point.

Question 31

Initial non-equivalence of groups

Answer 31

There should be no systematic differences between groups except for manipulation

Question 32

Ways to eliminate confounding variables due to participant characteristics

Answer 32

-Within-participants design -Random assignment -Matched pairs design

Question 33

What is a matched pairs design

Answer 33

Having matching pairs across groups of different characteristics, (e.g, every participant of a given age in one group, will have a matching participant of the same age in the other group.)

Question 34

Ways to eliminate confounding variables due to experimenters or procedures

Answer 34

-Ensure the conditions are as similar as possible except for manipulation (e.g, heating, noise) -Standardisation of the procedures (e.g., giving participants written instruction) -Randomisation of the orders of conditions performed (e.g, don't test participants in a condition before testing all participants in another condition)

Question 35

Differential attrition

Answer 35

When people leave one condition or treatment more than any other (drop out rates) - the data then becomes biased -Particularly important in longitudinal designs -Attrition expected in those designs, but problematic if it occurs in one condition more than the other

Question 36

Threats to validity in within-subjects design

Answer 36

-Primary disadvantage is order effects -Inescapable reality of many experiments

Question 37

Types of order effects

Answer 37

-Practice effects --> participants perform a task better in later conditions -Fatigue effects --> participants perform a task worse in later conditions because they become or bored -Habituation: Participants may become less sensitive to a stimulus through repetition (psychophsyiological experiment in particular)

Question 38

Counterbalancing

Answer 38

-Testing different participants in different orders, participants are assigned to orders randomly

Question 39

Validity of stimulus

Answer 39

-Stimulus needs to be carefully chosen -It determines validity and effectiveness of manipulation -Stimuli come in range of modalities -Chosen stimuli should represent a good range -Including a manipulation check may be useful

Question 40

Validity of measures

Answer 40

-Ensure method of measurement actually measures the construct you intend (construct validity) -Check if measure is sensitive enough to detect an effect (ceiling and floor effects)

Question 41

What can we do to increase internal validity

Answer 41

-Use appropriate stimulus and measures -Choose appropriate design: within vs between -Randomisation -Control group -Experimental control: holding other variables constant -Counterbalancing -Blinding -Standardisation of procedure

Question 42

External validity (Ecological Validity)

Answer 42

-Link between experiment findings and real world -Extent to which results have real-world relevance -Generalisability: how much our findings can be generalised to other groups of people across circumstances.