open science Flashcards
(15 cards)
what is statistical power?
probability of seeing a true positive
what is alpha (α)?
the highest acceptable risk of a false positive
typically 5%
what is publication bias?
researchers biased towards results which support their theories
why are significant results more likely to be published?
many journals value novelty and surprising results
non-significant results are often not published
non-significant replications are hard to publish
why are null results important?
a study, if well-designed, doesn’t fail, it tells the truth
what are examples of questionable research practices?
distorting the data to support the researchers’ hypothesis
typically say a result is significant if p < .05 but it’s almost always possible to get some result where p < .05
harking
what is HARKING?
hypothesising after results known
what is p-hacking?
way to cheat/lie with statistics
performing the analysis in different ways to get p < .05
only reporting the significant result
what is multiverse analysis?
run many possible analyses
see how many get a significant result
what are the problems?
significant results easier to publish, including false positives
many papers are underpowered, true positives are not seen
leads to many false positives in literature
how can the reproducibility crisis be solved?
transparency
open materials
open data
preregistered
what does transparency mean?
honest
accountable
what does open materials?
share the materials
exact instructions, program, stimuli
makes it easier for others to replicate
what is open data?
share the raw data
so other researchers can perform the analysis and see how other variables/analyses affect the results
what is preregistered?
plan the study in advance, including materials and planned analyses
prevents p-hacking and HARKING
researchers can compare preregistration to the final study
