Week 7 Flashcards
(32 cards)
1
Q
What is Test accuracy
A
Of all the people int he sample, this is the proportion of people who were correctly identified as either a) having or b) not having the condition
2
Q
What is prevalence
A
Of all the people in the sample, this is the proportion who have the condition
3
Q
What is sensitivity
A
Of all the people in the sample, this is the proportion who test positive
4
Q
What is specificity
A
Of all the people in the sample who do not have the condition, this is the proportion who test negative
5
Q
Screening tests
A
- It is not east to find a test that has a high sensitivity and high specificity
Choice has to be made:
Either:- High sensitivity and lower specificity (many true positives, little false negatives)
- Or
High specificity and lower sensitivity (many true negatives, little false positives)
6
Q
Evaluative tests
A
- Used to measure a variable of interest and to monitor the response to treatment
- Tests are often called outcome measures when used in this way
- Studies of intervention efficacy report outcome measures
- These same measures are used in practice to measure and monitor individual client or patient responses to intervention
- Although best research evidence indicates the amount of benefit on average a person can expect from the treatment there is no average patient
- Practitioners can therefore use outcome measures to quantify the benefits of treatment to an individual person or group
7
Q
Predictive tests
A
- Measure a variable or variables in the present and use that to predict something in the future e.g. risk of developing a particular health condition or probability that a particular event will take place in the future
- Helps determine the cause of a persons illness or problem
- Results are used to determine the likelihood that a person has or does not have a particular condition
- Diagnostic tests are used in differential diagnosis - the process of sorting out which of a number of possible conditions a person may have
- Tests with high specificity increase the probability or likelihood that the person has a particular condition
- Tests with high sensitivity decrease the probability that a person has a condition
8
Q
Screening tests
A
- Use to look for an uncommon but serious health condition
- Often people who do not have an identified health problem
- e.g. pap smear for cervical cancer allows early detection of a serious health problem
- In screening it is important to have a few false negatives ( a person with the condition testing negative for that condition)
- Designed to have maximum test sensitivity so people who test negative can be confidently said to not have the condition of interest
People with positive result undergo additional diagnostic tests to determine probability of the condition
9
Q
Key concepts of testing
A
- list the key differences between diagnostic and screening tests.
- explain the characteristics of a well-designed Level II diagnostic study.
- explain the following characteristics of a test and calculate the corresponding values:
- sensitivity and specificity;
- positive and negative predictive value;
- accuracy;
- prevalence;
- likelihood ratios.
- list the possible harms of testing.
10
Q
How can you tell if a study on diagnostic test accuracy is believable? Hierarchy?
A
- The hierarchy shows that the level 1 evidence is a systematic review of level 2 studies
- Level 2 studies are defined as ‘a study of test accuracy with an independent blinded comparison with a valid reference standard among consecutive persons with a defined clinical presentation
Table 2 outlines why these and other features of studies of diagnostic test accuracy are important in avoiding bias and how each type of bias impacts on the validity of the study’s estimates of test accuracy
11
Q
Index test
A
- In diagnostic studies of test accuracy a new test called the index test is compared against a reference standard test (also called gold standard test)
- The gold standard test is generally the best available for determining the truth about the presence or absence of the condition
- Alternatives are often developed if the gold standard is high risk, invasive, or expensive
- Gold standard tests themselves are not always 100 accurate
- Sometimes there is no true gold standard against which the index can be compared, in which case the researchers need to justify the reference criterion
A study should never include the index test as part of the gold standard criterion
12
Q
Contingency tables and test results
A
- A paper used the analogy of a number of people on trial for murder to explain the interplay between test results and the unknown truth about the presence of absence of a condition
- Ten men on trial: 3 committed murder 7 are innocent
- A jury hears the cases and: Finds 6 men guilty, 2 of the convicted are true murders, 4 wrongly in prison, 1 guilty man walks free
Sometimes the Jury’s verdict is wrong and in these cases, the guilty go unpunished and the innocent are incarcerated
13
Q
When administering a diagnostic test
A
- When administering a diagnostic test, it would be ideal if a positive test always meant the person had the health condition and a negative test always meant they did not
- However testes are rarely 100% accurate - some positive tests are false positives and some negative results are false negatives
- This means a positive test does not necessarily mean that the person has the condition and a negative test does not necessarily mean that they do not
- The process of diagnosis is a bit like a trial - the practitioner collects a range of evidence, weighs it up and makes a decision on the balance of probability that a particular health condition is most likely to be present
The person then treated for that condition
14
Q
What is positive predictive value
A
Of all the people in the sample with a positive test result this is the proportion who have the condition
15
Q
Negative predictive value
A
Of all the people in the sample with a negative test result this is the proportion who do not have the condition
16
Q
Interpreting and answering your research question
A
- Main question is: What is the probability that a person with a positive or negative test result actually has the condition for which they are being tested?
- The positive predictive value (PPV) and Negative Predictive Value (NPV) tell us the probability that a person in the study (or in a situation where the prevalence is identical in that study) who tests positive or negative has or does not have the condition
However because the prevalence of a condition can vary enormously the PPV and NPV often cannot be used to determine the probability of the person having the condition
17
Q
Rule of thumb for Likelihood ratios
A
1 = no change to the probability (pre-test or post-test) that the person has the condition
+LR> 10= a very large shift in probability (the person is more likely to have the condition)
+LR> 5 = A substantial shift in probability
-LR< 0.1 = a very large shift in probability (e.g. the person is substantially less likely to have the condition)
18
Q
Likelihoods ratio
A
- LR overcomes this problem
- The LR is the probability of a test result, in a person with the condition divided by the probability of the rest result in a person without the condition
A likelihood ratio can be calculated for a positive and negative test result - If the prevalence is used as the pre-test probability the PPV and NPV will give you the post-test probability
In clinical practice, you will more often be using your own diagnostic reasoning with regard to the pre-test probability for an individual patient
In simple terms, a likelihood ratio (LR) assesses how much a test result changes the likelihood of a patient having a disease compared to not having it. It compares the probability of a positive or negative test result in a person with a disease to the probability of the same result in a person without the disease
19
Q
Methods that can be used to calculate post-test probabilities
A
- Likelihood ratio nomogram
- Probability to odds conversion
- Posterior probability of disease calculator
Rough estimates of magnitude of change
20
Q
Summary of steps in using the nomogram to calculate post-test probabilities
A
- Mark your pre-test probability on the vertical line on the left of the nomogram
- Mark the LR of your selected test on the middle vertical line (for both a positive test and a negative test)
To determine the post-test probability, if the test is positive and if It is negative, draw a line through each set of two points
- Mark the LR of your selected test on the middle vertical line (for both a positive test and a negative test)
21
Q
To test or not to test
A
- If a clinician is already very sure that a person has a condition they should commence treatment for the condition
- Testing is indicated when the pre-test probability is between 20-80%.
- The exception is when the test is a screening test
22
Q
Screening tests
A
- Used to screen for conditions
- The pre-test probability is often low
- A good screening tests has high sensitivity (low rate of false negatives)
- A negative result in a test with high sensitivity means that there person almost certainly does not have the condition
- = SnOUT: Sensitivity, negative result, rules the condition out
- = SpIN: Specificity, positive result rules the condition in
The trade-off of achieving high sensitivity in a screening test is that there will often be a high rate of false positives - everyone with the condition tests positive, but so do a large number of people without the condition
23
Q
Population and clinic based screening tests
A
Population based screening
- Expensive so must be based on an evaluation of the evidence of the benefits, harms and costs of testing
- e.g. women for cervical cancer and prostate for men
Clinic-based screening tests
Some tests performed in particular clinical circumstance to rule out a condition that if left undetected can have serious consequences
24
Q
SnOUT and SpIN
A
- SnOUT: High sensitivity means a negative result rules out
SpIN: High specificity means a positive result rule IN
25
Spectrum Bias
Consecutive persons with a defined clinical presentation that reflects the clinical population
Prospective sampling of consecutive people who are representative of the clinical population on whom the test will be used provides an estimate of how the test will perform in practice. Studies that only include severe cases will overestimate sensitivity. Studies that include people known not to have the condition will overestimate specificity.
26
Incorporation bias
The reference standard is independent (does not incorporate) the index test
If classification of the target condition as present or absent depends at least in part on the results of the index test, then sensitivity and specificity will be overestimated.
27
Verification, workup or referral bias
Everyone who has the index test also has the reference test
If only the people who test positive on the index test go on to have the reference test the true and false negative results will be unknown. If those who tested negative on the index test do not have the gold standard, but are assumed to be true negative tests then sensitivity and specificity will be overestimated.
28
Double gold standard bias
Everyone gets the same gold standard test.
Sometimes unavoidable if the gold standard is invasive. If patients with a positive test get one gold standard, and those who test negative get another, unless the two gold standards are equally accurate, the results may be affected.
29
Observer bias
Independent, blinded comparison of the index test with a reference standard
If the person doing the reference (gold standard) test knows the results of the index test (or vice-versa) they are more likely to record a consistent result. This will overestimate the test accuracy.
30
LR+ and LR-
LR+ (Positive Likelihood Ratio):
Indicates how much more likely a person with the disease is to test positive compared to someone without the disease. A higher LR+ suggests the test is more useful for ruling in the disease.
LR- (Negative Likelihood Ratio):
Indicates how much more likely a person with the disease is to test negative compared to someone without the disease. A lower LR- suggests the test is more useful for ruling out the disease
31
Snout and Spin
SNOUT (Sensitive tests rule OUT the disease when negative):
A test with high sensitivity is good at identifying people who have the disease. If a sensitive test is negative, it suggests the disease is likely absent (ruled out).
SPIN (Specific tests rule IN the disease when positive):
A test with high specificity is good at identifying people who don't have the disease. If a specific test is positive, it suggests the disease is likely present (ruled in).
32
Spectrum, incorporation, verification, double gold standard, observer bias
Spectrum: the phenomenon where the performance of a diagnostic test varies across different patient subgroups or disease spectra. This means that a test that appears accurate in one population may not perform as well when used in another with different characteristics.
Incorporation: Incorporation bias, also known as review bias or verification bias, occurs in diagnostic accuracy studies when the results of the index test (the test being evaluated) are used in the determination of the reference standard (the "gold standard" for diagnosis). This can lead to an overestimation of the test's accuracy
Verification: when the decision to use a reference standard (gold standard) to confirm a diagnosis is influenced by the results of the index test (the test being evaluated)
Double gold standard: occurs in diagnostic test accuracy studies when different gold standards are used depending on the results of the index test
Observer: occurs when a person interpreting test results is influenced by their expectations or prior knowledge, potentially leading to inaccurate or skewed assessments.
