AAAA Therapy

Question 1

Objectives:

Answer 1

❖Introduce a framework for using population based research in clinical practice (Assess-Access-Appraise-Act)
❖Rehearse the framework for a question about the effectiveness of a treatment
❖Understand key methodological aspects of studies used to evaluate effectiveness of a treatment
❖Calculate and interpret common ‘effectiveness’ outcome measures

Question 2

What are the 4 elements of the aaaa framework

Answer 2

❖Assess: what type of healthcare question – what type of study?
❖Access: finding the ‘best’ evidence (validity/ trustworthy and relevance)
❖Appraise: evaluating the quality of the evidence & interpreting the results
❖Act: is this evidence relevant to my clinical practice. Should this evidence change my practice?

Question 3

Why is aaaa important

Answer 3

❖An essential professional and academic skill
❖Medical knowledge is continually evolving.
❖The medical profession frequently fails to use effective treatments even when they are available.
❖Keeping up to date is a lifelong commitment for every doctor.
❖You need to develop and use the skills to find, appraise and act on research evidence.

Question 4

What is evaluation bypass

Answer 4

Avoid evaluation bypass
Where interventions adopted with unevaluated procedure
With only enthusiasm, convictions, and commercial pressure
And no good evidence

Question 5

Example of RCT which was planned to address gap in evidence

Answer 5

❖Corticosteroids had been used to treat acute head injury for more than 30 years
❖MRC-CRASH Randomised Controlled Trial aimed to solve remaining uncertainty of benefit
❖International trial n=20,000 planned
❖Trial stopped early (n=10,000) – evidence of harm
❖Mortality at 2 weeks -21.1% with steroids -17.9% with placebo
❖Hundreds of thousands harmed (killed) internationally

Question 6

What study design is used for a effectiveness question

Answer 6

RCT

Question 7

Cause and effect their is a hierarchy of evidence. Three key characteristics that determine validity of study:

Answer 7

Casual agent - treatment
Effect - benefit to patient

1. Temporality - whether cause proceeds the outcome. RCT and cohort studies high in hierarchy
2. Experimental evidence - RCT less biased than observational studies
3. Evidence from several studies rather than one - systematic reviews are best

Question 8

What to do when assessing clinical question

Answer 8

• Effectiveness is a balance btw benefits and harm
• Potential side effects

Question 9

What are clinical components of an effectiveness question

Answer 9

❖Population: those who are eligible for treatment with the intervention / comparator
❖Intervention: New treatment
❖Comparator: Existing treatment / usual care (may be no
treatment)
❖Outcome(s): Mortality / morbidity / physiological measure (BP)
Formulating PICO is important for:
• Efficient searching for evidence in electronic bibliographic databases such as MEDLINE
• Assessment of the applicability of research to your patients

Question 10

What is access in the framework

Answer 10

How to find the best study
That are clinically relevant to the question

Question 11

How to access specific RCTs

Answer 11

❖Use the components of your question-specific PICO
❖Start with terms for Intervention of interest. If necessary to increase
specificity of the search combine with terms for the Population
❖Often more than one one term to describe question components which can be taken from:
-bibliographic database thesaurus’ (MEdical Subject Headings) -non MESH terms for the question component (free Text
terms)

Question 12

What are Boolean operators

Answer 12

❖Combine terms using Boolean operators:
-“AND” gives search results which include both search terms
-“OR” gives search results which include either one or other or both search terms

Question 13

What is appraise in the framework

Answer 13

Quality of the conduct of the study
Interpretation of the results

❖Does the study address a research question which is relevant to my clinical problem
(is there a match between my PICO & the study question?)
❖Did reserchers use the study design most likely to provide a valid answer to the clinical question
(for effectiveness was it an RCT?)

❖Was the study done well / is the study trustworthy ? (were steps taken to reduce bias?)
❖If the study was done well (valid), what were the results?
(direction, size, precision and statistical significance of results)

Question 14

What is bias

Answer 14

If pattern of deviation has a systematic pattern

Question 15

Key Features of an RCT

Answer 15

Prospective
Exposure (treatment) comes before we measure the outcome
New treatments/ comparator
Randomisation

Question 16

Why do we have a control group

Answer 16

Having a control group allows us to evaluate whether patients would get better anyway - the comparator treatment /no treatment

Question 17

Why do drug manufactures not allocate treatment

Answer 17

Patients have different abilities to benefit from treatment
Manufacturer may be inclined to select patients with a better prognosis to have new treatment
May make new treatment look better than actually is
Removes selection bias

Question 18

Why cant patient/ dr allocate treatment

Answer 18

Patients and doctor may choose for more severe patients to receive the new treatment
Skews the results and data
Imbalance between group with treatment and control
So removes selection bias

Question 19

What is randomisation

Answer 19

Every person has equal chance
Allocation decided by chance
Minimises the chance of selection bias
Minimised differences btw groups being compared at the start of the trail
At end of trail any differences observed should be due to differences in intervention and comparator only

Question 20

What is allocation concealment

Answer 20

❖Allocation concealment hides the allocation of trial participants from participants and those recruiting participants so that this knowledge cannot influence (as per previous slides) who actually receives which treatment. Minimises the chance of selection bias
❖How:
-a computer generated random sequence should be
performed distant to where the trial is being conducted
-where possible, treatment allocation should be concealed until its administration (sealed envelopes, similar packaging etc)

Question 21

Why can allocation concealment be hard

Answer 21

Hard to conceal interventions sometimes
E.g. physical activity

Question 22

What is the placebo effect

Answer 22

‘a beneficial effect produced by a placebo drug or treatment, which cannot be attributed to the properties of the placebo itself, and must therefore be due to the patient’s belief in that treatment’.

Question 23

Why is blinding important

Answer 23

To avoid measurement bias
And prevent performance bias

Question 24

What is measurement bias

Answer 24

❖Patients may offer socially desirable responses if they know they are in the intervention arm

❖Observers (those measuring outcomes) may have preconceived ideas about how effective a treatment is. This can influence how they measure subjective outcomes

Question 25

What is performance bias

Answer 25

❖Healthcare professionals may have preconceived ideas about how effective a treatment is. This can influence (consciously or subconsciously) how they deliver treatments, and may lead to participants in either the intervention or comparator arm receiving ‘extra’
❖Study participants may be disappointed about which arm they are randomised to which may lead to them behave differently, for example not engaging with treatment

Question 26

What is blinding?

Question 27

What are the levels of blinding

Answer 27

Who can be blinded:
- Participants
- clinicians delivering treatment
- researchers measuring outcomes
- statisticians undertaking analysis

Depending on how many types of participants are blinded:
- single blind
- double blind
- triple blind

Question 28

What are the 2 elements of follow up of participants

Answer 28

1. Follow up should be complete.
   -Patients may be lost to follow up at different points in a trial
   -Patients may be available for outcome measurement but are not adhering to the trial protocol (not taking/receiving allocated treatment)
2. Follow up should be long enough for outcome to be measured. Dictated by type of outcome.
   E.g.
   Infection - days/ weeks
   Recovery from cancer - years

Question 29

What scenario is least likely to undermine the validity of RCT

Answer 29

Where there is a similar loss from both treatment arms
Not about the total loss - about whether proportion of loss is balanced

Question 30

What is attrition bias

Answer 30

a type of selection bias due to systematic differences between study groups in the number and the way participants are lost from a study.

Question 31

What may be a common pattern of drop out in RCT

Answer 31

Drop out of patients with severe disease in control
Drop out of patients with mild disease as have more to loose due to side effects

Question 32

In an RCT, if patient drops out of intervention arm what should be the follow up

Answer 32

Intention to treat:
Patients who did not take the new drug should be analysed with the group allocated the new drug (ignoring the fact they did not take the new drug)
To uphold the validity of the RCT

If you exclude the patients - causes imbalances in arms. Undoes randomisation.

Question 33

Design features of RCT that minimise bias

Answer 33

Randomisation - selection bias
Allocation concealment - selection bias
Blinding - performance and measurement bias
Complete follow up
Intention to treat

Question 34

How to interpret results

Answer 34

❖Direction of effect:
-Do we want more or less of the outcome? -Have we got more or less of it?
❖Size of effect estimate ❖Precision of effect estimate
❖Statistical significance of effect estimate:
-Is any difference between groups likely to be real or due to
chance?
❖Clinical significance of effect
- If the difference is statistically significant, is it large enough to be clinically important (signficant)?

Question 35

What do you use to compare interventions

Answer 35

2 by 2 table

Question 36

What is risk difference

Answer 36

Percentage of patients that will benefit compared to control
First divide the patients that benefited by the total number of patients separately in both arms
Risk of benefit in intervention group minus risk of benefit in comparator group

Question 37

What is relative risk

Answer 37

Risk of benefit or harm
First divide the patients that benefited by the total number of patients separately in both arms
Ratio
Divide risk benefit of intervention group by comparator group

Question 38

What are 2 ways you can statistically compare risk benefits

Answer 38

Risk difference
Relative risk

Question 39

What is NNT number needed to treat

Answer 39

How many patients we need to treat with intervention for one more person to benefit compared to comparator
1/ risk difference

E.g.
4.5….. An extra 5 patients would need to be treated with chiro compared to physio to achieve one more patient with no pain at 3 months in the chiro group.
(NOTE: Unit of NNT is people and so need to round to a whole number – convention is to round up)

Question 40

What are the interpretations of results for relative risk

Answer 40

RR = no difference between groups
RR>1: more outcomes in treatment (chiro) group compared to comparator (physio) group.
RR<1 :less outcomes in treatment (chiro) group compared to comparator (physio) group.

Question 41

What are the interpretations of results for risk difference

Answer 41

RD = 0: no difference between treatment (chiro) and comparator
(physio) groups.
RD> 0 (+ve): more risk of outcome in treatment (chiro) group compared to comparator (physio) group.
RD<0 (-ve): less risk of outcome in treatment (chiro) group compared to comparator (physio) group.

Question 42

What are statistical tests

Answer 42

the probability of observing a false positive result/ it being due to chance

Hypothesis testing: p values
A p-value = 0.05 = 5% probability that the observed result is due to chance (a false positive result)
A p-value = 0.01 = 1% probability that the observed result is due to chance (a false positive result)

Question 43

How is precision of results calculated

Answer 43

Using confidence intervals
an estimation of the range within which the true size of effect lies

-EG a 95% CI for a RD of +22 (+18 to + 26) = if we repeated the trial 100 times, 95/100 times we would observe a RD between +18 and +26

If the RD lies within the range then we can deduce that it is statistically significant

Question 44

What is act in the aaaa framework

Answer 44

Whether we should change practice due to evidence that study presents

Question 45

How do consider whether to act

Answer 45

Refer back to pico
How close the study participants match the patients in healthcare
What is the cost of the intervention?
What other treatments are available? What is their cost?
Were all important outcomes (including harms) considered?