Open science

Question 1

Bem (2011)- “Feeling the Future”

Answer 1

-Task: 100 pps, 36 trials (positive, negative, romantic, erotic images)
-Key findings: hit rate above chance only for erotic images (53.1%, p= .01)
-Issues: suggests precognition, which is implausible
-Problem: used standardised statistical methods and was published- raises concerns about research practices

Question 2

Statistical power

Answer 2

Likelihood of detecting true effect

Question 3

Alpha level

Answer 3

Acceptable risk of false positive (typically 5%)

Question 4

Types of results

Answer 4

-True positives
-False positives
-True negatives
-False negatives

Question 5

Publishing bias

Answer 5

-Preference for significant novel, or surprising results
-Non-significant findings often unpublished
- Replications undervalued

Question 6

The ‘file drawer problem’

Answer 6

Non-significant findings often unpublished

Question 7

Importance of null results

Answer 7

Provide valuable insights from well-designed studies

Question 8

Questionable research practices

Answer 8

-HARKING
-Data manipulation

Question 9

HARKING

Answer 9

Hypothesising after results are known

Question 10

Data manipulation

Answer 10

Exploiting data analysis flexibility to find significance

Question 11

Researcher degrees of freedom

Answer 11

-Flexibility in:
*Stats tests
*Variable selection
*Outlier handling

Question 12

P-Hacking

Answer 12

-Performing multiple analyses to achieve p< .05
-Reporting only significant outcomes

Question 13

Consequence of P-Hacking

Answer 13

Increases false positives -> undermines trust in results

Question 14

Muñoz & Young (2018) multiverse analysis

Answer 14

-Less than 5% significant results from 1152 regressions
-Demonstrates how easy it is to find false positives flexible analysis

Question 15

2 major problems identified in research

Answer 15

-Publication favours significant results
-Studies often underpowered

Question 16

Reproducibility crisis

Answer 16

-Many studies fail to replicate
-Even experienced researchers struggle to reproduce published findings (Baker, 2016)

Question 17

3 solutions via open science

Answer 17

-Open materials
-Open data
-Preregistration

Question 18

Open materials

Answer 18

-Share study materials, instructions and stimuli
-Facilitates replication

Question 19

Open data

Answer 19

-Make raw data accessible
-Allows reanalysis and transparency

Question 20

Preregistration

Answer 20

-Outline study design and analysis plans before conducting the study
-Helps prevent p-hacking and HARKING