Evidence Based Medicine Flashcards
Learning Objectives
Distinguish between narrative review articles (non-systematic reviews), systematic reviews, and meta-analysis, and understand issues in using them.
Describe characteristics of a good clinical practice guideline, including a focus on clinical outcomes, use of best available evidence, and involvement of multi-professional groups and consumers in development.
What is evidence-based medicine?
“Evidence-based medicine is the integration of best research evidence with clinical expertise and patient values.”
- David Sackett
Combine: 1. Best Research evidence 2. Clinical Expertise & 3. Patient Values
What is “best research evidence”?
- Systematic Reviews of RCTs
i. The strongest evidence and statistics - Randomized controlled trials (RCT)
i. 2nd best evidence - Controlled observational studies
i. E.g., cohort, case-control, cross-sectional
ii. medium evidence - Uncontrolled observational study (case series)
i. okay evidence - Physiologic and animal studies
i. weak evidence - Unsystematic clinical observations, expert opinion
i. weak evidence
Intravenous streptokinase for acute myocardial infarction
a. Blood clots in critical vessels can cause MI
b. Streptokinase and other drugs break up blood clots
c. Between 1959 and 1988, 33 RCTs evaluated intravenous streptokinase versus placebo in patients hospitalized for acute MI
d. Total of 36,974 patients enrolled
* Large systematic review of Randomized control trials= awesome evidence
Forest Plot looking at these different Randomized Control Trials
a. Meta-analysis–> found benefit for streptokinase in preventing MI over many randomized controlled trials
b. Explain Forrest Plot
i. The results of meta-analyses are typically displayed as Forest plots, where each line represents a single trial, including the reference, often the year of publication, and the number of subjects in the intervention and control group.
ii. Next is the point estimate and confidence interval from that trial.
iii. If the horizontal line overlaps 1.0, then the trial did not have statistically significant results. When the point estimate is to the left of the line, the intervention group has a better outcome.
iv. When the point estimate is to the right of the line, the control group has a better outcome. The dark black dot at the bottom of the plot is the combined results of all the trials together.
v. The left side of the Figure shows that the effect of treatment with streptokinase on mortality was favourable in 25 of the 33 trials, but in only six was statistical significance achieved.
* Summary: The overall pooled estimate of treatment effect given at the bottom significantly favoured treatment.
Use of Human Albumin
in Critically Ill Patients
a. In critically ill patients, serum albumin concentration inversely related to risk of death
b. Human albumin solutions used for shock, burns, and illnesses associated with hypoproteinemia
c. Human albumin costs 30 times more than saline
d. Is it beneficial? Is it cost-effective?
Human Albumin Study-Large Summary of study
a. A systematic review of cohort studies meeting specified criteria estimated that for each 2.5 g/l decrement in serum albumin concentration the risk of death increases by between 24% and 56%.
b. The association persists after adjustment for other known risk factors and pre-existing illness, and some commentators have suggested the possibility of the albumin molecule having a direct protective effect.
c. Partly as a result of the association between serum albumin and mortality, Licensed indications for human albumin solution are the emergency treatment of shock and other conditions in which restoration of blood volume is urgent, the acute management of burns, and clinical situations associated with hypoproteinaemia.
d. Compared with other colloidal solutions and with crystalloid solutions, human albumin solutions are expensive.
i. Volume for volume, human albumin solution is twice as expensive as hydroxyethyl starch and over 30 times more expensive than crystalloid solutions such as sodium chloride or Ringer’s lactate.
e. Because of the high cost and limited availability of human albumin, it is imperative that its use should be restricted to the indications for which it has been shown to be effective.
f. To quantify the effect on mortality of human albumin solution in the management of critically ill patients with hypovolaemia from injury or surgery, burns, and hypoproteinaemia, researchers conducted a systematic review of randomised controlled trials.
What the results showed for albumin studies
Summary: Found that albumin treatment actually lead to higher rate of harm, when looking at a large meta-analysis of albumin treatment
i. each RCT by itself looked to have varied results.
a. The line down the middle represents a relative risk or odds ratio of 1.0. In this forest plot, the size of the box indicating the point estimate varies with the size of the trial, with larger trials with smaller variance having larger boxes.
b. Larger trials receive more weight in the quantitative analysis than do small trials.
c. In this example, you can see that nearly all the confidence intervals overlap an odds ratio of 1, indicating that the result is not statistically significant.
d. *When all results are combined, albumin was found to be actually harmful, that is, the placebo controls had a better outcome than the patients given albumin.
Why EBM matters
Identification of best available evidence and integration of evidence into practice has the potential to:
i. improve health and well-being
ii. Avoid harms and conserve resources
Systematic Review
a. Summary of the best available evidence to address a focused question
b. A systematic review is a type of literature review that collects and critically analyzes multiple research studies or papers.
i. A review of existing studies is often quicker and cheaper than embarking on a new study. Researchers use methods that are selected before one or more research questions are formulated, and then they aim to find and analyze studies that relate to and answer those questions.
ii. Systematic reviews of randomized controlled trials are key in the practice of evidence-based medicine.
Systematic Reviews versus Traditional Literature Reviews
a. Literature reviews, like all research, are subject to selection and information bias
b. Systematic reviews use standard methods designed to reduce bias
i. try to reduce bias
Sources of systematic error (bias)
in literature reviews
How the review author:
-defined the study question
- looked for studies
- decided what studies to include
- decided if the studies were any good
- decided what comparisons to analyze and what outcomes to report
- interpreted the results
Systematic vs Traditional Review
Benefits of Systematic Review:
a. Systematic Reviews are scientific investigations in themselves.
b. Well Formulated Question: that is, the specific purpose of the review is identified & stated.
c. Comprehensive and Explicit Data Search: Sources and methods of the citation search are identified, and there is an attempt to identify all relevant literature.
d. An Unbiased selection and Abstraction Process – that is, explicit guidelines are used to determine what material is included in and excluded from the review.
e. Critical Appraisal of data – The material in the review is assessed for methodological validity.
f. Appropriate synthesis of the data – the information is systematically integrated. Data limitations and inconsistencies are described. The information may be integrated and weighted or pooled quantitatively (I.e., through meta-analysis).
i. A summary of pertinent findings is provided.
Traditional Review: Broad, unspecified, variable in all factors
Could a statin lower my risk of having a heart attack?”
"John the patient"
Professor at CU Boulder
60 years old
Well controlled hypertension (BP 130/80)
Never smoker
No fam hx heart disease
No other medical history
Total cholesterol 210,
HDL 55
What research to seek out:
- Seek out Summary and Guidelines
i. see if there are guidelines - If no guidelines, then look at Pre-appraised research.
i. These are systematic reviews that have been brought together - Non-appraised research
i. these are clinical trials by themself
ii. can be risky, as seein with the albumin study
Hierarchy of Evidence
a. Is there an evidence-based practice guideline that gives me the information I need to optimize John’s care?*
b. If not, is there a systematic review that answers the question of whether a statin could lower John’s risk of having a heart attack?
Cochrane Library: “Statins for the Primary Prevention of Cardiovascular Disease”
i. Meta-analysis was found from randomized clinical trials
Critical Appraisal of Systematic Reviews
- Are the results valid?
i. do you believe them? - Are the valid results meaningful?
i. is there an actual benefit - Are the valid, meaningful results relevant to my patient / my practice
Critical Appraisal of Systematic Reviews
Validity
Are the results valid?
Is this a systematic review of the best available evidence?
Clearly focused question
Inclusion criteria specify appropriate study designs
Does it describe a comprehensive and detailed search for relevant trials?
Were the individual studies assessed for validity?
A. Clearly Focused Question
Important for a Systematic review
a. The specific purpose of the review is identified & stated
b. The question specifies:
i. Population studied
- Age, gender, health condition, activity, setting
ii. Intervention given
- Drug, surgical procedure, diagnostic test, program, regulation, engineering change, policy, law
iii. Outcomes considered
Condition, illness, behavior
Unfocused Question
Example of bad question for Systematic Review
What is new in treating breast cancer?
Too vague, no specific intervention, population, or outcome
Formulating a research question
a. Poorly focused questions lead to unclear decisions about what research to include and how to summarize it
b. Well-formulated questions guide:
i. determination of final criteria used to select studies to include in the review
ii. what data should be abstracted from included studies
Clearly Focused Question
How effective are antiepileptics for preventing epilepsy following traumatic head injury?
- Population: people with traumatic head injury
- Intervention: anti-epileptics
- Outcome: epilepsy
B. Comprehensive and Explicit Data Search
a. Search strategy based on explicit criteria developed from the focused question
b. Attempt to identify all relevant literature
c. Sources and methods of the literature search are identified
Define inclusion and exclusion criteria
P (Types of Population)
I (Types of Intervention(s))
C (Types of Comparisons, controls)
O (Types of Outcome(s))
S (Types of Study designs)
Types of Study Designs
a. Included studies should be the best available evidence, based on the Evidence Hierarchy, to address the review question
b. Appropriate studies for inclusion
i. RCTs for treatment, diagnostic tests
ii. Observational studies for hazardous exposures, prognosis
Locate and select studies
a. Comprehensive search strategies to find all relevant studies, to avoid bias
b. Bias = systematic errors that cause results to be consistently distorted in one direction because of nonrandom factors
i. Bias is a Distortion of the results
c. Two main types in systematic reviews:
- Publication bias
- Location bias
Looking at studies and what gets published
A= All Studies Conducted
+, - and null findings, statistically significant and non-significant results
B=All Studies Published
+, statistically significant
C=all studies that are identified in a systematic review – location bias
*Location Bias—> looking for studies that have a positive result, will have a location bias
Location Bias
Location Bias: Bias in
a. The publication of research findings in journals with different ease of access or levels of indexing in standard databases, depending on the nature and direction of results.
b. There is also evidence that, compared to negative or null results, statistically significant results are on average published in journals with greater impact factors, and that publication in the mainstream (non grey) literature is associated with an overall greater treatment effect compared to the grey literature
Bias in Systematic Reviews
a. Publication Bias: Tendency for published studies to differ systematically in their results from unpublished studies
i. +, significant studies more likely to be published
b. Location Bias: Tendency for high profile, widely disseminated studies to differ systematically in their results from low profile, less widely disseminated studies
i. +, significant studies easier to find
How to reduce Publication and Location Bias in Systematic Reviews
a. Avoid language restrictions
b. Search more than one electronic database (hundreds exist)
c. Search other types of documents (abstracts, reports, letters, dissertations)
d. Check references of other studies, books
e. Contact experts or organizations
f. Contact trialists for unpublished outcomes
C. Unbiased Selection and Abstraction Process
a. Explicit guidelines used to determine what studies to include and exclude
i. Participants, Interventions, Comparisons, Outcomes, Study designs
b. Screen identified records for eligibility based on stated inclusion criteria
i. Best practice: two investigators independently screen and assess relevance
D: Critical appraisal of identified studies
Quality assessment of each included study using standardized criteria
Internal Validity
a. Extent to which each study’s results are valid for circumstances being studied
b. Beneficial effects of trials are likely to be overestimated if:
i. Inadequate allocation concealment
ii. Lack of blinding
iii. Inadequate randomization generation
iv. High dropout rates
(Concealment of group allocation means it is not possible to tell to which group the next enrolled subject will be assigned, and therefore characteristics of the subject cannot influence group assignment.)
Quality Assessment: How?
a. Inclusion criteria and screening
i. E.g., only include double-blind RCTs with >80% follow-up
b. Component assessment of included trials
i. Allocation concealment
ii. Randomization sequence generation
iii. Blinding
iv. Loss to follow-up (dropouts)
c. Scales for quality assessment
Q1: Is this a systematic review of best available evidence?
Look for:
a. A clearly focused question
i. population, intervention, outcomes
b. Inclusion of the appropriate sort of studies to address the review question
i. RCTs for treatment, diagnostic tests
ii. Observational studies for hazardous exposures, prognosis
Statins for the Primary Prevention of Cardiovascular Disease”
Critiquing the study
P: People without CVD
I: Statins
O: CVD
Types of studies: RCTs comparing treatment with statins for at least 12 months with placebo or usual care; follow-up at least six months
Question #2: Does the review describe a comprehensive and detailed search for relevant studies?
a. Look for:
- Which bibliographic databases were used
- Follow up from reference lists
- Personal contact with experts
- Searches for unpublished studies
- Search for non-English language studies
b. Consider:
- Explicit use of inclusion and exclusion criteria
- Use of two independent reviewers
- Standardized abstraction of study data
