9. Open Science

Question 1

What is p-hacking?

Answer 1

A form of manipulating statistical analyses in order to produce a significant result. Researchers can do this by selectively choose which data to analyse, or performing multiple statistical analyses until a significant result is found. Leads to overreporting of false positives.

Question 2

What is the reporting bias/file drawer effect?

Answer 2

Selectively reporting some results of a study and omitting others, could be as a result of publication bias

Question 3

What is publication bias?

Answer 3

Studies with positive results are more likely to be published. Leads to overreporting of false positives and overestimating of effect sizes in aggregated results.

Question 4

What is pre-registration?

Answer 4

Publically posting a time-stamped record of the study protocol prior to data collection/analysis, with a commitment to make results public. Has the benefits of no p-hacking, HARKing, and can unambiguously distinguish between exploratory and confirmatory hypothesis testing.