Module 3 Flashcards
(25 cards)
What are the 4 strategies (questions) to assess the study evidence and quality?
- what type of evidence is this?
- does this research apply to my topic?
- how much is the result affected by bias?
- what are the effects?
What is the process of evidence-based practice? (similar to 5 A’s)
- ask question
- find best evidence
- appraise the evidence
- implement
- evaluate outcomes
What is the difference between a systematic review and a narrative review?
A narrative review is not an exhaustive or structured review of the literature, it is more susceptible to bias and does not systematically evaluate the quality of included studies according to pre-determined criteria
A systematic review is a detailed, comprehensive and structured and critical review of all research studies that address a particular clinical issue.
What is an example (type of study) of systematic review?
meta-analysis
What does PICO stand for?
Population/patient
Intervention
Comparison/control
Outcome
What are some questions you could ask for each letter of the PICO model?
P:
how do you define your population?
who are the relevant people, patients, participants?
I:
What is the intervention or exposure of interest?
C:
What is the alternative or standard practice?
who is the control population?
O:
What is the relevant outcome?
Who are the key people involved in peer review process and what are their tasks?
- journal staff: they oversee receipt of manuscript, communication between author and reviewers and take care of formatting
- editorial board: read/review papers, select reviewers, monitor quality, recommend actions to editor
- editors: make final decisions
- reviewer: provide reviews of manuscripts, make recommendations concerning publication
What are the attributes of a good reviewer?
- under 40 yo
- training in epid/stats
- quality institution
- 3h review time
What is the peer review process?
- the reviewer is contacted by the journal
- the reviewer is given the authors, title, abstract and time frame
- the reviews agrees or declines to review the paper
- reviewer receives paper
- reviewer performs review
- reviewer submits review to editors
- editors examine reviews, obtain additional reviews if needed, and make decision
- decision goes to author with comments from reviewers
- reviewer may be informed of the decision
What are the contents of the peer review process?
- review form
- comments to editor
- comments to author
What are the 3 steps to summarize an article?
- What was the purpose of the study (intro)
- Study details (methods)
- Study findings (results)
What are the criteria to look at when evaluating research?
- importance
- originality
- relevance to readers
- usefulness to readers and to patients and stakeholders
- truth
- wow factor
- clear and engaging
What is a journal’s impact factor?
total # of times its articles were cited during the 2 previous years/ total # of citable articles in the journal during those 2 years
What are the criteria by which authors of scientific papers are justified?
- substantial contributions to the conception or design of the work, acquisition, analysis or interpretation of data work AND
- drafting the work or revising it critically for important intellectual content AND
- Final approval of the version to be published AND
- Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved
What are the attributes of scientific writing?
- formal writing style, highly technical
- not overly descriptive or flowery, sentences not complicated, paragraphs are short
- goal is to clearly convey information
What are the differences between a journal article and review article?
Journal:
- describes original research conducted by authors
- utilizes a structured framework
Review:
- integrates, summarizes and provides ideas for extending upon prior research on a particular topic
- authors study existing literature
Describe the manuscript components?
Title: capture the message, main thrust of paper
Abstract: <300 words, concisely summarize paper, stand-alone, indexed searched
intro: what did you study and why? general -> narrow -> hypothesis: what are knowledge gaps or contentions in field?
methods: how you did it, can the reader repeat your study now?
Results: what you found but no interpretation
Discussion: what does your work mean? compare/contrast with others, implications? strengths/weaknesses? future work? conclusions?
ref list: do not cite a paper if you have not read it, follow a journal style
acknowledgements: thank the helpers, collaborators, funders
other stuff: how chose journal? follow guidelines carefully
What are the 10 aspects of effective data?
stored preserved accessible discoverable citable comprehensive reviewed reproductible reusable successful data
What is the spoof study?
molecule X from lichen species Y inhibits the growth of cancer cell Z
Results/conclusion of the study:
1. cancer cells grow more slowly in test tubes when treated with higher concentrations of a molecule (results of the study show the opposite)
2. cells treated with higher doses of radiation to stimulate cancer radiotherapy
outcome: many journals still accepted the article
Why are most studies false?
- large studies are more likely to yield true results yet we publish studies with small sample sizes
- willingness to publish small effect sizes: with a smaller effect size, findings are less likely to be true
- flexibility in designs, definitions, outcomes and analysis reduce chances that results will be true
- fishing expeditions to generate new hypothesis or explore unlikely correlations
- financial and other interests and prejudices reduce the likelihood that results will be true
- competition in research to produce positive findings, especially in hot fields
What is the amgen study? What was the main issue with it?
efforts to characterize genetic alterations in human cancers -> increased understanding of molecular mechanisms
Hope this would translate to more effective drugs yet research results had low success
not able to reproduce 47 out of 53 publications
What are the points of reproducibility crises?
- statistical standards:
- avoid making decisions solely on the p-value
- be more rigorous and use p<0.001 rather than p<0.05
- present the confidence interval - data handling:
- researchers should make their data available
- researchers should use born-open data - research practices:
- researchers should pre-register their protocols, filling in advance with an appropriate organization (allows to control the researcher who changes mid way in a study based on preliminary results)
- adopt standardized schemes to outline methods and materials - pedagogy:
- fields that rely too heavily on statistics to draw conclusions should be better educate on the mis-use and mis-understanding
- teach more holistic approaches as well as reasoning approaches to analyze data - universities:
- when profs go up for tenure, they should be required to adhere to best-practices for research methods - professional associations and journals:
- establish regular evaluation of disciplinary norms
- journals should make peer review process even more transparent and rigorous
- journals should only publish pre-registered studies
- all disciplines should establish a journal devoted to publishing negative findings - scientific industry:
- industry should advocate practices that minimize irreproducible research
- work with academic to formulate standard research practices and protocols to promote reproducible research - private philanthropy:
- fund scientists effort to replicate earlier findings
- fund researchers who strive to develop better methods
- funding university chairs in reproducibility studies
- establish a prize for most significant negative result in various disciplines
- improve journalism that continues to uncover the reproducibility crises - government funding
- fund scientists effort to replicate earlier findings
- fund researchers who strive to develop better methods
- prioritize funding for researchers who pre-register their plans and make data/methods open access
- adopt new NIH principles for funding reproducible research
- more funding to be broaden statistical literacy - government regulation
- ensure new regulations needing scientific justification only use research that meets strict reproducibility standards
- establish committees to determine which regulations are based on reproducible research - federal legislation
- pass a secret science reform act to prevent agencies from making regulations based on irreproducible research
- strengthen information quality act
- fund programs to broaden statistical literacy - state/provincial legislation
- reform K-12 curricula to include courses on statistical literacy
- use funding and oversight to encourage universities to add and strengthen statistical literacy - government staff and judiciary
- government officials should hire trained staff in statistics and reproducible research to advise them on scientific matters
- courts should ensure that sound science is used in judicial decision making
- set approaches to overturn precedents based on irreproducible science
- relevant courses in la school
What is the premise and design of the scientific rigor?
premise:
- what is the basis for your proposed research study?
- strengths and weaknesses of past research
- have you scanned the entire literature or have a myopic/biased view? are you systematic? critical? CRAAP?
design:
- foundation for achieving robust and unbiased results
- strict application of scientific method to design, method, analysis, interpretation and reporting of results
- standards? sample size estimator? randomization? blinding? replicates? inclusion and exclusion criteria?
What are biological variables?
- sex, age, weight, underlying health affect health and disease
- variables often ignored -> incomplete understanding
- explain how relevant biological variables are factored into research
- key now is sex
