Week 8. Evaluating Diagnostic Literature Flashcards Preview

572 - Evidence-Based Practice > Week 8. Evaluating Diagnostic Literature > Flashcards

Flashcards in Week 8. Evaluating Diagnostic Literature Deck (33):
1

Validity

- Is it true? Can I believe it? Are the outcome measures trustworthy and accurate?
- Extent that a measure assess what it is intended to measure.

2

Applicability

If valid and important, can/should I apply it to my patients

3

Use of Diagnostic Tests

PT’s have increased access to DI but it should not replace clinical assessment/tests
E.g. Shoulder imaging — physicians order shoulder imaging to facilitate referral to surgeon but after prolonged wait periods, surgeon refers patient to PT

4

Diagnosis Research Goals

1. Evaluate whether a test gives additional information about presence/absence of a condition
2. Evaluate whether clinical test provides similar information as an invasive or radiological test
3. Evaluate whether a diagnostic test is able to distinguish between patients with and w/o a specific condition
4. Avoid invasive tests/x-ray exposure, more carefully define injured structures/tissues to customize treatment

5

Clinical Prediction Rules (CPR)

Ottawa Ankle Rules

- Used for diagnosis
- A rule or model that tries to identify the best combination of S&S, and other findings for predicting the probability of a specific outcome

OAR
- sensitivity: 96-99%
- specificity: 26-48%
if negative, low chance of fracture
(point tenderness at posterior edge (of distal 6 cm) or tip lateral malleolus. point tenderness at posterior edge (of distal 6 cm) or tip medial malleolus. inability to weight bear (four steps) immediately)

6

Level of evidence in diagnostic study design:

Why can't RCT be used in Dx studies

- RCT cannot be done in Dx studies as all subjects must undergo both tests
Level 1 evidence in Dx studies
- Cross-sectional
- Cohort study designs

7

Methodological Issues in Diagnostic Research

- Gold Standard Test

Inappropriate gold standard/reference test

8

Methodological Issues in Diagnostic Research

- Verification Bias

Verification Bias:
results in test influence the decision to have the gold standard test

9

Methodological Issues in Diagnostic Research

- Selection/referral bias

Selection/Referral Bias:
Evaluation done in a Population with a high prevalence of disease or investigators pick study participants

10

Methodological Issues in Diagnostic Research

- Measurement Bias

Measurement Bias
- Testers are aware of gold standard tests results which bias outcome
- Outcomes for what constitutes positive/negative are not well-defined
- Testers unable to complete diagnostic test properly

11

Sensitivity (SnNout)

Sensitivity: likelihood of a +test in presence of disease (true positive rate)

SnNout:
- a negative result on a highly sensitive test is a good way to rule out people who don’t have the condition
- Example: airport security, if highly sensitive will pick up all kinds of metal, so no buzz = no metal, but lots of false positives, but you don’t miss things

12

Specificity (SpPin)

Specificity: likelihood of a -test in the absence of disease (true negative rate)

SpPin: a highly specific test will not falsely identify people has a condition, a positive result on a highly specific test is likely to accurately detect the presence of a condition
- Example: airport security, if airport sensor is turned down, it would be highly specific (metal = buzz)

13

Positive Prediction Value

Likelihood of disease in the presence of +test

14

Negative Predictive Values

Likelihood of not having a disease in the presence of a negative test

15

Positive/Negative predictive values table

- Rows calculate?
- Columns calculate?

Rows = Predictive Values
Columns = sensitivity/specificity

(TP)True+ (FP) False+ Total who test positive
(FN)False- (TN) True+ Total who test negative

Total w. Total w/o
Disease. Disease

16

Accuracy

Accuracy = (a + d) / (a+b+c+d)

= (TP + TN) / (TP + TN + FP + FN)

17

Sensitivity calculation (true positive rate, TPR)

Sensitivity = a / (a+c)

- true positive divided by total number with disease.
- this is the probability of positive test if subject has disease, also called true positive rate

18

Specificity Rate (True negative rate, TNR)

Specificity = d/ (b+d)

- computed as true negatives divided by total number without disease
- defined as probability of negative test if subject does not have disease; true negative rate (TNR)

19

Positive Predictive Value

PPV = a / (a+b)

- computed as true positive divided by total number that tested positive
- defined as probability of disease if subject has a positive test; true negative rate (TNR)

20

Negative Predictive Value

NPV: d / (c+d)

- true negative divided by total number that tested negative
- probability of no disease if subject has a negative test

22

Specificity vs. NPV vs. -LR

Specificity (d / b+d)
- do not have disease
- probability of negative test

NPV (d / c+d)
- negative test
- probability of no disease

-LR: probability of a negative test result given the presence of the disease and the probability of a negative test result given the absence of the disease, i.e.

23

Which aspect is dependent on prevalence of disease?

PV are dependent on prevalence of disease, while sensitivity/specificity are not
- PV are meaningless out of context of prevalence
- Sensitivity and Specificity are dependence on diagnostic threshold; more consistent BETWEEN studies

24

Sensitivity and specificity
- more reliable INTER- or INTRA?

- most consistent between studies
- diagnostic threshold for a sp diagnostic test defined as min or max requirement to obtain a positive result

25

Receiver Operator Characteristic Curves (ROC Curves)

- 3-way relationship between sensitivity, specificity, and diagnostic threshold
- curve shows trade-off between sensitivity and specificity with changing diagnostic thresholds


ROC values
0 = terrible
1 = ideal

26

Positive Likelihood Ratio

- ratio indicates?
- probability that a person?
- Value indicates?

Sensitivity / (1-specificity)

- ratio of true positive rate to false positive rate
- probability of a person with a positive test has the disease
- larger numbers indicate higher likelihood of disease

27

Negative LIkelihood Ratio

- ratio indicates?
- probability that a person?
- Value indicates?

NLR: (1 - sensitivity) / specificity

- ratio of true negative rate to the false negative rate
- used to determine the probability with a negative test does not have the disease
- smaller numbers indicate higher likelihood of NO disease

28

LR+ of 7.29 and LR- of 0.166
If individual takes test for the disease, we can update their probability of disease by multiplying odds by the likelihood ratio

If test is positive, updated odds of disease: (1/99) x 7.29 = 0.0736
If test is negative, updated odds of disease (1/99) x 0.166 – 0.00168
Odds of disease increases from 1% to 7.4% with positive test and decreases from 1% to 0.16% with a negative test

29

+LR. -LR.
> 10. < 0.1
5-10. 0.1 - 0.33
3-5. 0.34 - 0.99
<3. > 1

Almost conclusive
Useful
Marginally Useful
Likely not important

30

Clinical Utility of DI Statistics: Sensitivity/Specificity

- most common reported values
- can calculate LR from these values

31

Clinical Utility of DI Statistics: PV

- limited useful ness because less stable estimates (depends on population tested/prevalence of disease)

32

Clinical Utility of DI Statistics: LR

- Most clinically useful because they contain both sensitivity/specificity values in 1 ratio

33

Example: A new ‘special test’ for the shoulder has been developed to test for a presence of a rotator cuff tear
Want to compare results of the new test to a known standard

A = 50
B = 10
C = 15
D = 25

Accuracy?
Sensitivity?
Specificity?
PPV?
NPV?
+LR?
-LR?

Accuracy = 75/100
Sensitivity = 77%
Specificity = 71%
PPV = 83%
NPV = 62%
+LR = 2.7 (likely not important)
-LR = .32 (may be useful)

34

Sensitivity vs. PPV vs. LR+

Sensitivity (a / a+c): probability of a positive test if subject has disease (TPR)
- they have the disease
- probability of positive test recognizing


PPV (a / (a+b): probability of disease if subject has a positive test
- they have a positive test
- probability of actually having disease

LR+ (sensitivity / (1-specificity)): probability of a positive test result given the presence of the disease and the probability of a positive test result given the absence of the disease,