Chapter 14

Question 1

3 key components of the best practices/open science

Answer 1

transparency
reproducibility
replicability

Question 2

reproducibility

Answer 2

reproducing identical results from the same data

Question 3

replicability

Answer 3

replicating results generated from older data by collecting new data through similar procedures

Question 4

what does replication give to a study

Answer 4

credibility

Question 5

3 types of replication

Answer 5

direct replication
conceptual replication
replication-plus-extension

Question 6

direct replication

Answer 6

the original study is repeated as similarly as possible to determine whether the original effect is found in the new data

Question 7

conceptual replication

Answer 7

the same research question and same conceptual variables but different operationalizations

Question 8

in replication plus extension in what 2 ways can you replicate the original study

Answer 8

1. add another level to an existing IV
2. add another variable (makes it a factorial design)

Question 9

what does a meta-analysis yield?

Answer 9

a quantitative summary of a scientific literature/ an average of the effects from all studies (published and unpublished( on the same variables

Question 10

scientific literature

Answer 10

series of related studies conducted by different researchers who have tested similar variables

Question 11

limitations to meta-analyses

Answer 11

null and opposite effects are rarely published so a meta analysis might overestimate the true effect size (file drawer problem)

Question 12

solution to the file drawer problem of meta-analyses

Answer 12

actively seek unpublished data and use social media forums

Question 13

origin of the replication crisis

Answer 13

only 39% of a random sample of 100 studies published in journals had been replicated

Question 14

recommended rxns to the replication crisis

Answer 14

1. ask why replication studies might fail
2. ask what the best practices are to improve reproducibility

Question 15

why might a study fail to replicate?

Answer 15

-if direct replication was used when it doesn’t make sense to use it
-if the researchers relied on only 1 replication study
-questionable research practices

Question 16

best known QRPs

Answer 16

underreporting null fx
p-hacking
HARKing
using small samples

Question 17

how does underreporting null fx influence readers?

Answer 17

makes people think that the effects are stronger than they actually are

Question 18

p-hacking

Answer 18

when researchers try running different statistical analyses or computing their data differently than they originally intended (in hopes of obtaining a significant p value, it’s not done intentionally but they can become biased and not be aware they’re doing it)

Question 19

HARKing

Answer 19

hypothesizing after the results are known; misleads readers about the strength of the evidence

Question 20

why can a small sample size be problematic

Answer 20

the study’s estimate is usually imprecise and not replicable because it doesn’t take many extreme variables to greatly influence the data set

Question 21

best practices for scientific studies

Answer 21

pre-registration
power analysis
report all analyses
report all variables measured
report all conditions

Question 22

pre-registration

Answer 22

preregister the study’s methods, hypotheses and statistical analyses online BEFORE DATA COLLECTION

Question 23

power analysis

Answer 23

determines the adequate sample size according to the design; done before submission to ethics committee

Question 24

what 2 factors are considered in external validity

Answer 24

-how well the results can generalize to a population of interest
-how the sample was selected (random?)

Question 25

is a study fundamentally flawed if it does not use a representative sample?

Answer 25

NO

Question 26

is a large sample more representative than a small sample?

Answer 26

NO

Question 27

ecological validity

Answer 27

the extent to which a study’s tasks and manipulations are similar to what people experience in real life

Question 28

mundane realism

Answer 28

replicating what people experience in real life

Question 29

theory-testing mode

Answer 29

testing association or causal claims to support a theory, and external validity is NOT a priority

Question 30

famous example of theory-testing mode theory

Answer 30

contact comfort theory

Question 31

generalization mode

Answer 31

generalizing findings from the sample to other populations or contexts, external validity IS a priority

Question 32

cultural psychology

Answer 32

a special area of generalization mode, some cultures see things differently than other cultures (ex: Muller-Lyer illusion)

Question 33

WEIRD sample

Answer 33

western educated industrialized rich and democratic samples. be aware of studies that are primarily based on these

Question 34

experimental realism

Answer 34

the extent to which participants experience authentic emotions and behaviors

Question 35

how is the importance of a study not solely determined by external validity and mundane realism

Answer 35

when a study is capable of allowing participants to experience authentic emotions (experimental realism)