Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

research methods 2 > Effect Size > Flashcards

Effect Size Flashcards

1
Q

why is effect size useful?

A

experiments intend to find the effect of your IV on a DV

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

what does a p value tell us?

A

measures the probability of obtaining the observed results, assuming that the null hypothesis is true
- tells us whether an effect exists

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

what does the p value not tell us?

A

the size of the effect

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

what does effect size measure?

A

indicates the proportion of the variance explained

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

what effect size is used for correlation and regressions?

A

R and R^2

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

what effect size is used for T-tests?

A

Cohen’s d

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

what effect size is used for one way ANOVAs?

A

Eta Squared

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

what effect size is used for factorial anovas (where the effect is made up of different variables)?

A

Parietal Eta Squared

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

how do you calculate cohen’s d?

A

differences between the mean as a function of (/divided by) the standard deviation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

what is Cohen’s d?

A

standardised score representing the difference between the group mean

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

d = m1 - m2 / 𝜎

A

D is the difference in means -> scaled by the standard deviation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

what is cohen’s d an example of?

A

a standardised score (conducted with means rather than individual scores)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

what are the benefits when we scale by standard deviation in cohen’s d?

A
  • D does not dependent on a sample size (you can compare cohen’s d for a small pilot study and the full experiment because it’s standardised and divided by the standard deviation)
  • allows you to compare studies effect with other studies in literature
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

what is cohen’s d for ANOVA (where there are multiple groups)?

A

difference between largest and smallest group mean scaled by standard deviation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

cohen’s d for anova’s calculations

A

d = mean(max) - mean (min) / 𝜎

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

what are the assumptions for cohen’s d for ANOVAs calculation?

A
  • standard deviation is assumed to be constant across groups
  • only true for samples with met ANOVA assumptions (homogeneity of variance assumption -> assessed using Levene’s test)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

There are various conventions to calculate the overall standard deviation for an ANOVA. What are these?

A
  • averaging group SD
  • taking the smaller SD (more conversation)
  • pooling the variance (looking at different combinations and using the outcome of that)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

what is eta squared?

A

proportion of variance explained by your experiment
(variance that’s explained by your variance / total variance of the model)
- used for one-way ANOVAs

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

why is eta squared only used for one way ANOVAs?

A

cause only one variable and one effect size

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

what is the equation for eta squared?

A

Eta(n)2 = SSeffect / SStotal

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

how to calculate eta squared?

A

divide the sum of squares for the effect by the total sum of squares

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

what is partial eta squared?

A

proportion of variance that is uniquely explained by each variable -> finding out how strong / big the effect of your variables
- factorial ANOVAs used (tells you in estimates of effect size ANOVA table)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

why is partial eta squared used for factorial ANOVAs?

A

more than one variance and effect size

24
Q

what does p value measure?

A

the significant percentage of variance that is explained

25
Q

what is the equal for partial eta squared?

A

Partial eta2 = SSeffect / SSeffect + SSerror(/residual)

26
Q

what is partial eta squared??

A

ratio of variance associated with an effect, plus that effect and its associated error variance

27
Q

what are (partial) eta squared scaled between?

A

0 (none of the variance) and 1 (100% of the variance)

28
Q

how do you report effect size?

A

alongside other statistical information:
F(df)=…., p=…., n2 =…

29
Q

can you have a significant effect, yet a really small eta squared?

A

yes -> in this case, think about how meaningful this is and what else could be having an effect on your dependent variance (if there is only a small significant effect)

30
Q

what is power analysis?

A

used to determine whether your design is appropriate and may find the effect you’re looking for
- can be confident about the result you are looking for -> by how powerful your design is

31
Q

when is a power analysis run?

A

usually before you start collecting data

32
Q

what does the ‘power’ of a statistical test do?

A
  • ability to detect an effect when it is actually there (powerful enlightenment to be able to see an effect)
  • ability of a test to correctly reject the null hypothesis (can confidently say when zooming in there is nothing there)
33
Q

what does power dependent upon / influenced on?

A

sample size, criteria for significance and effect size

34
Q

sample size

A

the more participants = increased power and chance of finding a significant effect (if there is one)
* useful because if we know how many subjects we need to detect an effect size at a given power

35
Q

what is usually considered a good level of power?

A

around 0.8
- can help you understand that you may need like at least 17 subjects to achieve a significant effect at a power level of 0.8
[minimum number of participants to test in order to find an effect)

36
Q

(power depending on) effect size

A

find an effect we know is there in reality
* smaller effect size need more participants to achieve a higher power -> interaction between number of participants, power and effect size [if we know at least two of these, then we can estimate the other that we are trying to find]

37
Q

(power depending on) power of significance [alpha level, a]

A

as you increase/decrease alpha level, it will influence how powerful your design is

38
Q

what do we want to do about power?

A

make sure we have enough power to detect an effect
( be confident in that result, or be confident that their isn’t a result -> reject null hypothesis it indeed shows a null result)

39
Q

what are the dangers of an underpowered study?

A
  • unpowered (too few participants)
  • lack of power to detect effect -> type 2 error (false negative)
  • increased chance of type 1 error (false positive) -> can’t be confident if the same size is smaller etc.
  • If we estimate power across many published studies it is often worryingly low
  • e.g. Button et al (2013) estimated the median power of studies is around 0.08-0.31
  • Low power explains failure to replicate
  • current replication crisis in Psychology (and other disciplines) -> and therefore we don’t know who’s right
40
Q

power analysis also carries ethical issues. what ethical issues can this carry?

A
  • Expensive
  • Inconvenient
  • Boring
  • Uncomfortable
  • Painful
  • Dangerous
  • Even Fatal (animal research)
41
Q

what is a requirement for ethical approval now?

A

demonstrating your study has sufficient power
* no justification for running a study if it doesn’t stand a reasonable chance of being informative esp if painful, dangerous etc
- wasting people’s time is unethical too including causing someone discomfort if you are not confident in your result

42
Q

what should you look into, in order to be relatively confidence about your findings?

A

pilot studies

43
Q

what are four ways in which we can estimate effect size?

A
  1. Guess
  2. Pilot Study
  3. Find Previous Research
  4. Find or Conduct a Meta-analysis
44
Q
  1. Guess
A
  • good if there’s much literature on your phenomenon
  • you can guess / estimate the effect size of your experiment by using cohen’s heuristic
    BUT it’s not terribly satisfying nor super informative :/ [garbage in, garbage out model -> based on nothing] -> but can be used to say that even if the effect is tiny, we had enough participants to find an effect
45
Q
  1. Pilot Study
A
  • Fewer participants to estimate the effect size
  • does not matter if study comes out significant -> still use it to get an estimate of r
  • you can use this estimate to project how many participants you should test
  • BUT estimates of effect sizes are largely unreliable with such small samples :/
46
Q
  1. Find previous research
A
  • literature search -> studies investigating similar phenomenons
  • use their results to work out an expected effect size for your experiment
  • these studies will not use the exact same size :/
    BUT a-priori power estimate are not exact
  • You can’t predict the future!
  • Better than guessing!
47
Q
  1. Find or conduct a meta-analysis
A
  • literature search
  • if there are many previous studies you can calculate an average effect size across all of their results
  • common in drug trials
  • often a meta-analysis will have already been published that you can take effect sizes from or work them out yourself from them
48
Q

what is a problem with power analysis?

A

GIGO: Garbage in -> Garbage Out technique
* If you make up the numbers you enter -> what you get out is not meaningful
* estimates are not exact because you can’t predict the future
* for really complex designs (i.e. factorial), it is unlikely you will have sufficient precision in your estimates of effect size
- main effect & interactions
- power analysis may not be meaningful -> gets lower and lower when you get through really complicated designs -> should be interpreted with caution

49
Q

what are the three types of power analysis?

A
  • A priori
  • Sensitivity
  • Post Hoc
50
Q

A priori

A

Calculate how many participants are required for a study (-> given effect size, power, alpha)

51
Q

Sensitivity

A

Calculate minimum required effect size detectable (-> given power, sample size and alpha)

52
Q

Post-Hoc

A

Calculate observed power (-> given effect size, alpha and sample size)

53
Q

How to conduct a power analysis using G*Power

A
  • Test Family: F Tests (everything ANOVA giving you an F ratio) [obvs T instead for T tests]
  • Statistical Tests -> different options for the ones we are looking at -> we will typically look at ANOVA (select the one which is most appropriate)
  • Select the Type of Power Analysis (Post Hoc, A Priori or Sensitivity)
  • Enter the study design in the input parameters section
  • find the output -> i.e. if only 0.3, not significant really
54
Q

what will the critical F value tell us?

A

the minimum F ratio required to reach significance

55
Q

Powerful analysis is useful in which situations?

A
  • estimating how many participants you should recruit
  • determining the power of a study design
  • working out the minimum effect size a study could detect reliably
56
Q

*may be stuff on cheat sheet online which is helpful

A

research methods 2 (8 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions