Week 10. Systematic Reviews

Question 1

h-Index

Answer 1

author-level metric that measures productive and citation impact of the publications of a scientist/scholar

“a scholar with an index of h has published h papers each of which has been cited in other papers at least h times”

Question 2

Journal Impact Factor (IF)

Answer 2

measure reflecting the yearly average number of citations to recent articles published in that journal.

• used as a proxy for the relative importance of a journal within its field
• higher IF deemed to be more important that lower ones

Question 3

Types of SR

Answer 3

• literature/narrative
• scoping review
• state-of-the-art review
• systematic review

Question 4

Narrative review

Answer 4

• summarize different 1* studies to draw conclusions with interpretations informed by the reviewers’ own experience, existing theories and/or models
• allows experts to synthesize and interpret information and form opinions/hypothesis
• most susceptible to bias

Question 5

Example of narrative review

Answer 5

• like a mini-CPG
• practice points
• up-to-date overview from clinical experts

Question 6

Scoping review

Answer 6

• Preliminary assessment of potential size and scope of available research literature
• Aims to identify the nature and extent of research evidence (usually including ongoing research)
• Casts a “broad net”

Question 7

example of scoping review

Answer 7

• Informs future research
• identifies gaps in knowledge
• inventories tools, methods, or approaches

Question 8

Systematic Review

Answer 8

literature review that involves “a comprehensive search for relevant studies on a specific topic, and those identified are then appraised and synthesized according to a predetermined and explicit method.”

Question 9

Example of systematic review

Answer 9

• Rigorous approach
• Least susceptible to bias
• Unfortunately, often recommend more research is needed

Question 10

Relationship of review

Answer 10

Narrative review < quick scoping review < rapid evidence assessment < full systematic review

Question 11

Types of SR

Answer 11

1. Intervention SR
2. Diagnostic accuracy SR
3. Prognostic SR
4. Risk/Etiology SR
5. Measurement tool properties SR.
6. Qualitative SR

Question 12

where does SR fit in the 5S Hierarchy of Evidence

Answer 12

Lowest level of secondary: Synthesis

Secondary (filtered resources)
1. Systems 
2. Summaries
3. Synopses 
4. Syntehsis 
Primary (unfiltered resources)
5. Studies (RCT, Cohort, Case control, Case series, case report)

*6. expert opinions: classical textbooks

Question 13

Why SR?

Answer 13

• Explicit methods reduce bias which means that conclusions are likely to be reliable
• When a meta-analysis is done, it helps increase the precision of the answer provided by many smaller studies (i.e. shrinks CIs)
• Saves the time of healthcare providers

Question 14

What are SR used for?

Answer 14

• Recommendations and guidelines
• Policy decisions
• Research agendas
• Individual patient care
• Patient decisions

Question 15

What is a meta-analysis

Answer 15

• Meta-Analysis – a statistical method to synthesize or pool data from multiple studies.
• Systematic reviews ≠ meta-analysis
• Systematic reviews may use meta-analysis to quantitatively combine studies
• Meta-analyses are not always systematic (can also be done as easily on an arbitrary grouping of studies)

Question 16

Components of SR

Answer 16

1. Focused clinical question (PICO)
2. Comprehensive search for evidence
3. Explicit inclusion and exclusion criteria for studies
4. critical appraisal of the included studies
5. systematic synthesis (if quantitative then also referred to as a meta-analysis)

Question 17

Steps to conducting a SR

Answer 17

1. Assemble a team
2. Ask the right question
3. Conduct a systematic and comprehensive search
4. Screening and selecting studies
5. Data extraction
6. Quality assessment (QA) of included studies
   7a. Narrative Data synthesis
   7b. Meta-Analysis

Question 18

Steps to conducting a SR: (1) Assemble a team

Answer 18

1. Assemble a team

- Content experts, SR method experts, statistician, searcher/librarian

Question 19

Steps to conducting a SR: (2) Formulate Question

Answer 19

2. Ask the right question

- PICO, determines the scope of project and facilitates subsequent steps of the SR

Question 20

Steps to conducting a SR: (3) Conduct a systematic and comprehensive review

Answer 20

• find all relevant studies that answer question

- consult/work with librarian/expert searcher

Question 21

Steps to conducting a SR: (4) screening and selecting studies

Answer 21

• Search results are compiled into a database (e.g. Refworks) and duplicates are removed.
• 2 people review ALL abstracts and select articles to minimize mistakes and bias.
• Need a clear process for resolving disagreements

Question 22

Steps to conducting a SR: (5) Data extraction

Answer 22

• Scan each included study for the data you would need to compare and synthesize studies (populations, settings, methods, treatment effects, adverse events)
• Create spreadsheets/tables
• Ideally 2 independent reviewers extract data from each study to reduce bias/errors

Question 23

Steps to conducting a SR: (6) Quality Assessment of included studies

Answer 23

• Not all evidence is high quality
• Quality = absence of bias
• Validated QA tools available
• Different QA tools for different study designs. (e.g. Cochrane risk of bias tool favoured method for QA of RCTs)
• 2 independent reviewers do QA for all studies

Question 24

Steps to conducting a SR: (7a) Narrative Data synthesis

Answer 24

• Narrative synthesis is done with non-quantitative data or because the studies are too heterogeneous
  Organize results as appropriate (e.g. study design, study quality, differences in population (mild, severe)
  Provide conclusions for research and clinical practice

Question 25

Steps to conducting a SR: (7b) Meta-analysis

Answer 25

- A statistical methodology to pool effect sizes across studies - Essentially creates a new larger, more precise study - Studies must be clinically homogenous in terms of interventions and outcomes - Create graphs called Forest Plots or Blobbogram

Question 26

Critical Appraisal of SR

Answer 26

AMSTAR = A MeaSurement Tool to Assess systematic Reviews’

Question 27

Credibility and Applicability: 4 questions before applying results of SR to patient

Answer 27

1. Was the SR done in a rigorous way? 2. Are the included studies high quality = low bias? 3. What are the results? 4. Are results applicable to my patient?