Critical appraisal 4

Question 1

What is ROC?

Answer 1

Receiver operator curves/ characteristic

Question 2

What does a ROC show?

Answer 2

graphical plot of true positive rate (sensitivity) against false positive rate (specificity)

useful for describing how good a diagnostic test is

the greater the area under the curve, the better the diagnostic test is

also helps provider an suitable cut-off value for a result

Question 3

What is intention to treat analysis?

Answer 3

Intention-to-treat analysis A method of analysis for randomized trials in which all patients randomly assigned to one of the treatments are analyzed together, regardless of whether they completed or received that treatment, to preserve randomization.

reflects real-world application of intervention

Question 4

What are pros of intention to treat analysis?

Answer 4

reflects real-world application of intervention - so if you get a positive conclusion, this is even more significant than a comparable RCT that has drop out

simplifies outcome/ bias as no patients are excluded

preserves baseline balance between groups

preserves sample size

Question 5

What are cons of intention to treat analysis?

Answer 5

estimate of intervention effect is conservative due to dilution of drop outs/ non-compliance

heterogeneity introduced when noncompliants, dropouts and compliant subjects are mixed together

Question 6

What are these alternatives to intention to treat analysis

As treated analysis

Answer 6

As treated analysis -
records classifies RCT participants according to the treatment that they received rather than according to the treatment that they were assigned to
subject to confounding in the same way as an observational study

Question 7

What are these alternatives to intention to treat analysis

Per protocol/ on treatment analysis

Answer 7

‘Per protocol/on treatment” analysis -
only includes individuals who adhered to the clinical trial instructions as specified in the study protocol
subject to selection bias due to cross-over and loss to follow up
per protocol analysis may be appropriate when analysing adverse events in drug trials, as it can be argued that side-effects of actual treatment received is clinically relevant

Question 8

What are these alternatives to intention to treat analysis

Modified intention to treat

Answer 8

Modified intention to treat -
allows the exclusion of some randomized subjects in a justified way (e.g. patients who were deemed ineligible after randomization or certain patients who never started treatment)
definitions used are irregular and arbitrary; consistent guidelines for its application are lacking
a subjective approach in entry criteria may lead to confusion, inaccurate results and bias

Question 9

What is a Type I error?

Answer 9

Type I error is a false positive

Question 10

What is a Type II error?

Answer 10

Type II error is a false negative

Question 11

How to calculate sensitivity?

Answer 11

Probability of those with the disease, who test positive

TP/ TP + FN

A sensitive test helps rule out a disease when a test is negative

Question 12

How to calculate specificity?

Answer 12

Probability of those without the disease, who test negative

measures ability to detect absence of the condition

TN/ TN + FP

A specific test helps rule in a disease when positive

Question 13

What is the PPV?

Answer 13

probability that the disease is present, when the test is positive

PPV = TP/ TP + FP

this is different to sensitivity, as is depedends on the population prevalence of the disease

Question 14

What if NPV?

Answer 14

probability that the disease is absent, when the test is negative

NPV = TN/ TN + FN

this is different to specificity, as is depends on the population prevalence of the disease

Question 15

What are overall differences between sensitivity/ specificity and PPV/ NPV?

Answer 15

Sensitivity/ specificity - characteristics of the tests themselves. High values suggest a good test. They enable you to rule conditions in or out, but cannot definitely diagnose a condition

PPV/ NPV - main use is clinical relevance of a test

PPV/ NPV depend on prevalence of the disease in population. But can tell you how the likelihood of a test diagnosing a specific disease. As prevalence decreases, the PPV decreases, as a positive is more likely a false positive. NPV increases as there will be more true negatives

Example -
Ebola test is 90% sensitive. Your patient tests positive. It would be incorrect to say this patient has a 90% chance of having Ebola. Sensitivity only helps you rule out a disease when it is negative. If your test is 90% sensitive, and is negative, you can probably exclude that disease

If Ebola test has 90% PPV, then you can say there is a 90% chance of this positive test result meaning that the patient has 90% chance of having Ebola

Question 16

Example questions during critical appraisal

What research designs were used in this study?

How were subjects recruited?

What are the advantages of this study over other studies?

What are the potential limitations and possible confounding factors in this study?

Explain how you would design another study to confirm the findings of this paper

Question 17

Example questions during critical appraisal

Explain the principle of testing by minipools.

What are the advantages and disadvantages of testing by minipools?

How are the sizes of minipools determined?

What factors affect the transmission of hepatitis E virus from a viraemic blood donor?

Based on the data from this paper, is it justified to introduce blood donor screening for hepatitis E virus?

Explain the arguments, both for and against.

If blood donor screening for hepatitis E is to be introduced to the UK, what method should be used?

If blood donor screening is not to be introduced to the UK, what other strategies should be used to mitigate the risk of transfusion
- transmitted hepatitis E

Question 18

Example questions during critical appraisal

There are 700,000 live births in England and Wales per year. What is the burden of congenital CMV infection and its complication?

What were the recruitment criteria in this study?

Would there be any possibility of recruitment bias?

What were the long term benefits of 6 month of valganciclovir treatment to symptomatic newborns with congenital CMV infection compare with 6 weeks of treatment?

Are there any cautions before accepting these findings?

Was CMV viral load an important determinant of outcome? Suggest a reason why this is the case?

To what patient groups is the finding of this paper not applicable?

Answer 18

1000-2000 births every year have cCMV
5% of these will have long term sequeale