Levels, sample size and power

Question 1

what is data collected at?

Answer 1

• data collected at different levels

Question 2

what do levels of data analysis depend on?

Answer 2

• depends on design and measures

Question 3

what are the two different levels of data analysis?

Answer 3

• within a person
• between people

Question 4

what are four examples of data within a person?

Answer 4

• response on questionnaire
• reaction time
• blood concentration
• physiological measurement

Question 5

what are four examples of data analysis between people?

Answer 5

• individuals in a sample
• samples in a population
• classes/ cohorts in a school
• districts in a city

Question 6

what is the most common level of data analysis?

Answer 6

• between people is most common

Question 7

how many repetitions should you use?

Answer 7

• no hard rule
• more is better

Question 8

what are the limitations of large sample sizes?

Answer 8

• time, money and resources limited
• participants’ effort, endurance, boredom, will- power and kindness are limited
• benefit increases only with square root of N

Question 9

how do you double statistical power?

Answer 9

• quadruple number of repetitions

Question 10

what do you do with all the repetitions?

Answer 10

• take averages
• one per design level

Question 11

what is the alternative with all the averages?

Answer 11

• take average of differences

Question 12

what do you do with all the averages of averages?

Answer 12

• analyse them
• use t- test, correlation, ANOVA, GLM

Question 13

how many levels of data analysis is there?

Answer 13

• three
  1-3

Question 14

what is level 1 of data analysis? - give an example

Answer 14

• within- person, within- measurement
  e.g., using 100 individual heart beats to estimate the heart rate

Question 15

what type of data is level 1?

Answer 15

• raw data

Question 16

what is level 2 data analysis?- give an example

Answer 16

• within- person, within condition
  e.g., using 10 repetitions of each condition to estimate pre/ post HRs

Question 17

what data/ statistic is level 2?

Answer 17

• summary data
• descriptive statistics

Question 18

what is level 3 data analysis? - give an example

Answer 18

• between people
  e.g., using 16 athletes in each group to estimate intervention effect

Question 19

what statistics is level 3?

Answer 19

• inferential statistics

Question 20

what do all general linear model statistical tests assume?

Answer 20

• assumes your data are independently sampled

Question 21

how is the assumption not true for a heart rate experiment?

Answer 21

• heart rate now is not independent from your HR 5 seconds ago

Question 22

what happens to the measurements from the same individual?

Answer 22

• correlated or dependent

Question 23

what should you use for a valid GLM test?

Answer 23

• only use one value per person and condition

Question 24

how can you use all the level 1 data?

Answer 24

• use a multi- level mixed model regression (MLMM)

Question 25

what designs usually need less data?

Answer 25

- within- subject (repeated) designs

Question 26

what is statistical power?

Answer 26

- the probability that you will find a significant result

Question 27

what does statistical power increase with?

Answer 27

- increases with the square root of N

Question 28

what do you need to set in statistics?

Answer 28

- set an arbitrary level of significance

Question 29

how do you evaluate sample size?

Answer 29

- evaluate sample size in context of other things

Question 30

do you need more than 20 for parametric GLM stats?

Answer 30

- NO, false assumption - t-tests were created for N= 4 to 10 - more is usually better

Question 31

what should you do if the study has a small sample size?

Answer 31

- poor criticism - if you know it is too small just tell me what size it should be

Question 32

what is assumed about the effect when there is a significant result?

Answer 32

- assuming that there is a real effect - whatever your hypothesis, you assume it is exactly true

Question 33

what is assumed about the sample when finding a significant result?

Answer 33

- that it is exactly as big as you say it is

Question 34

what do you need to specify? what is the equation?

Answer 34

- specify an effect size> R2, f, cohen's d Cohen's d = t / sqrt (N)

Question 35

what does statistical power assume about other statistical assumptions?

Answer 35

- assuming that other statistical assumptions are true - independent sampling - similar variance, independent residuals, etc

Question 36

what is the simplest case when calculating the effect size?

Answer 36

- a t- test

Question 37

how do you guess the sample size?

Answer 37

- independent samples t-test - calculate N required - calculate Cohen's d

Question 38

are there many hard rules in statistics?

Answer 38

- no - very few hard rules

Question 39

what is sample size not easy to do?

Answer 39

- not easy to assess or criticise

Question 40

how many datapoints should be used?

Answer 40

- more datapoints is usually better

Question 41

how can sample size be analysed?

Answer 41

- re effect size

Question 42

what can power analysis help with?

Answer 42

- helps you design a study