Critical Appraisal of Clinical Trials

Question 1

Explain the concept of a systematic review.

Answer 1

• literature review focused on a single question
• identifies, appraises, selects and synthesises high quality evidence relevant to that question
• well-defined criteria for review (cf narrative review)
• review of both clinical trial and observational data
• highest level of evidence

Question 2

What is a meta-analysis?

Answer 2

• statistical aspect of a systematic review
• analysis of combined data from multiple studies
• derive pooled (weighted-average) effect size

Question 3

What is the purpose of a meta-analysis?

Answer 3

• 4 purposes:
  i. increase power
  ii. resolve uncertainty
  iii. improve estimates of effect size (precision)
  iv. answer other questions

Question 4

How are studies identified?

Answer 4

• clear clinical / research question
• PICOT parameters:
  P = population
  I = intervention (including dosing and duration)
  C = comparator
  O = outcome (strict, objective definition)
  T = time

Question 5

Explain some sources of data.

Answer 5

• PUBMED, MEDLINE, EMBASE, CENTRAL, CINAHL, DARE
  – standard (eg MESH) terms where possible
  – include all relevant spelling
  – limits - eg RCTs
• reference lists
• others: ‘grey literature’ (eg AIHW), www

Question 6

What is meant by exclusion and inclusion criteria?

Answer 6

• in line with the question being asked and the PICOT parameters
• It also lines with other parameters (such as sample size)
  
  • be wary not to introduce bias

Question 7

What is meant by selecting studies?

Answer 7

Selecting Studies:

• undertaken by (at least) 2 people independently
• read all abstracts - sometimes hundreds!
• apply inclusion / exclusion criteria
• obtain full papers
• assess for quality

Question 8

What is the CONSORT form?

Answer 8

The CONSORT statement looked at a revised recommendation for improving the quality of reports of parallel-group randomized trials. There is a checklist of requirements for randomised trials known as the CONSORT checklist.

Question 9

What is involved in undertaking a meta-analysis?

Answer 9

Undertaking a Meta-Analysis:

• statistical software - eg, STATA
• purpose-designed software - eg, RevMan 5

Question 10

What are some key statistical issues associated with conducting a meta-analysis?

Answer 10

Key Statistical Issues:
- outcome: weighted average effect size
- relative measure -eg, RR, OR
- absolute measure - eg, mean difference
- weighting of individual studies
- sample size
- inverse variance
[ the greater the variance from the mean of the studies included, the lower the weighting it will have in the overall meta-analyses. We do not want studies that are heterogeneous]
- heterogeneity - variability in effect sizes
- chi-square test
- fixed-effects or random-effects modeling

Question 11

What is meant by heterogeneity?

Answer 11

Heterogeneity:

• validity of a meta-analysis relies on whether component studies similar enough to be pooled
• similarity:
  
  • statistical - effect sizes and variances
  • non-statistical - PICOT
• non-statistical heterogeneity cannot be objectively assessed

Question 12

What is meant by combining studies?

Answer 12

Combining Studies:
- “Meta-analyses should only be considered when a group of trials is sufficiently homogeneous in terms of participants, interventions and outcomes to provide a meaningful summary.”

• “Decisions concerning what should and should not be combined and inevitably subjective and are not amenable to statistical solution but require discussion and clinical judgement.”