Publication and its Biases Flashcards
Know the different traits, pros and cons, of subscriptions vs. open access publishing models
Subscription: pay to access it
Pros: better quality control
Cons: Paywall, consumer pays for the research
Open Access: author pays and everyone can access it
Pros: everyone gets access to it. consumers don’t pay
Cons: Quality control goes down, allowing for less reliable research
Be able to define what makes a low-quality for-profit journal a problem for scientific knowledge, and why they are flourishing recently
Is is a problem as they accept low quality research, which may have issues with the p value. This may contribute to false positives. Due to the research not being sound. They are flourishing because it is easy to get published,
Be able to define the seven traditional criteria for publishing psychology studies
- Methods allows you to make a conclusion from the date
- Alternative explanations are either ruled out or acknowledged
- Conclusion is appropriately limited given the methods
- Study makes a new contribution to the literature
- Study makes a positive contribution (shows that something exists)
- Strong statistical evidence for the conclusion
- More generalisable studies are seen as stronger
What are the methods of “gaming”, novelty of contribution, and why is it a problem?
Authors sometimes overhype their novelty. Direct replications have a hard time being published because they are not novel. Leads to papers stating a greater effect being found than there actually was.
What are the methods of “gaming”, Positive contribution of research, and why is it a problem?
Non-significant finds have a hard time being published. Thus leading to found effects and effect sizes being exaggerated in published literature, leading to inaccurate results.
What are the methods of “gaming”, strong statistical evidence (in terms of p-value) and why is it a problem?
P-hacking and selective reporting to make evidence looks more conclusive than they actually are.
What are the methods of “gaming”, strong generalisability, and why is it a problem?
Theoretical research is published more over applicable research, may lead to research not actually being generalisable. Bias for WEIRD samples.
What are the 9 criteria in the checklist to identify suspicious and mistaken conclusions from studies?
- the source
- conflict of interest
- alternative explanations
- validity of manipulations and measures
- validity in the real world
- multiple studies and their p-value pattern
- evidence against selective publication policies
- power that is adequate to test reasonable effects
- the right way to go about critiquing the generality of a sample
Why are negative results considered less worthy of publication than positive results?
Researchers are more interested in reading significant results, so journals tend to publish this, as negative results may be less interesting and could be due to issues with how you conducted the study.
Know the implications of the Turner study of clinical trials for the existence of publication bias
Due to all positive results being published, while very few negative ones were, this came across as the tested drug being 94% effective. However, when in tacking into account the negative studiers, it dropped to 51% effectiveness. We see positive more than negative results, causing this issue.
Know how and why scientific reporting in the popular media can further increase positive results bias and exaggerate effects
Only positive results are reported by the media, with corrections happening rarely.
Understand the definition of “legitimacy” and the difference between scepticism of a source journal, and a finding being right or wrong
Legitimacy: The paper is written in good faith, it is carried out using appropriate methodologies and it is taken seriously by the relevant scientific community
The higher the prestige of a journal and the more citations it has, the more credible the findings of the paper are. While less prestigious journals will hold less credibility due to having a less vigorous reviewing process.
