Diagnostic Evaluation Flashcards
What is a test?
- A process or device designed to detect something, often a disease but could also be:
- Clinical sign, substance/agent, tissue change, or body response
- These tests include anything that collects information or a status: physical exam, clinical signs, post-mortem change, laboratory finding (hematology, serology, biochemical, histopathology, etc.)
What is the difference between accuracy and precision?
You need to know whether a test is either accurate or precise or both.
Accuracy tells you the true value: how good this test is at telling you the true state.
Precise means it is repeatable aka you will always hit that mark perfectly.
- = Consistently get the wrong value
- = Every time you run the test, a lot of variability but on average it is correct. No inherent bias but every time you run the test, there is a bit of fluctuation
- = Consistently giving wrong value and also on average all over the place.
What is Analytical Sensitivity and Specificity?
- Analytical Sensitivity: lowest concentration the test can detect this particular agent or toxin, or whatever you are measuring. This is a LAB based measure.
* ‘Limit of Detection’ (LOD) (Are you able to detect a certain level in the sample?)- e.g. PCR: # DNA|RNA copies present in a sample
- Analytical Specificity: degree of cross-reactivity with non-target agents. Lab based question. If you are measuring a certain agent, that it truly is that agent.
- Highly specific tests detects only the target agent
What do diagnostic test evaluations require?
Two requirements:
* The test will detect diseased animals correctly.
* The test will detect non-diseased/healthy animals correctly.
These are two separate things.
One group of diseased animals and another group of healthy animals. You apply your test to this group if diseased animals, what do you expect to have?
Then you go into a completely naive population, are completely healthy from that particular disease in the other group. What do you expect?
What can happen here?
You will expect all of them to be positive.
You will expect them to be all negative
Diagnostic tests are not perfect.
Sensitivity formula
How many of them test + that are diseased.
p = Probability that,
T+ = you test positive
| = given that
D+ = they are diseased.
Specificity formula
How many of them test -.
p = Probability that,
T+ = you test negative
D+ = they are disease negative/healthy animals.
What is the gold standard in terms of diagnostic tests?
‘Gold standard’
- Rare, but some tests are VERY accurate. Sometimes expensive, or hard to do.
* A test or procedure that is absolutely accurate! …but no test is perfect!
* As close as we can get to accurate
* Examples:
1. Histophathological and microbiological examination of the small intestine is regarded as gold standard for Johne’s disease in cattle
2. Immunofluorescence antibody test (iFAT) for rabies (checking for viral particles in brain + inflammation)
What are the components of diagnostic testing?
Often several components are needed:
1. Identification of agent via: Culture, +/- PCR or molecular sequence confirmation, Antigen-detection tests (e.g. FAT)
2. Histological changes consistent with this disease
3. Presence of specific antibodies
4. Clinical signs or history of exposure to agent
Very hard to find and clearly define what a truly diseased animal is.
If you want a truly healthy animal, you have to go to?
Healthy (non-diseased) animals often come from naïve populations
* Regions/Farms known to be ‘free’ of certain agents
Sensitivity and specificity
Description of the diagnostic test performance
* Not always reported by the manufacturer of the
diagnostic test
* Often independent studies report Se/Sp values in
various populations. You go out and try to validate the test and publish your results.
* Theoretically varies between populations
* E.g. based on agent strains, cross-reactions with
endemic agents is possible, etc.
Determined by carrying out specially designed
studies
* Part of formal ‘Diagnostic Test Validation’ pathways
* Guidelines provided by the World Organisation for
Animal Health (OIE)
A lot of diagnostic tests are on what type of scale?
Give examples.
A continuous scale
* Cut-off values are used for +ve and –ve status
- Cut point = anything below/above this value is considered diseased/healthy. Assumption here is that there is a distribution of values for a diseased set of animals.
Optical Density – ELISA (proportion/concentration of Ab in sample).
Biochem:
- Glucose (mg/dL)
- ALT, ALP
- Creatinine, BUN
- …etc.
Hematology: cell counts
Where do you put your cut-point?
This is a sliding scale and you have to find a balance.
Should do a fairly good job most of the time, except for the small portion of animals in the purple AKA false positive and false negatives
This is an example of?
Gold standard
This is an example of?
A very bad diagnostic tests. Some exist that are that bad.
What is this an example of?
What does each letter stand for?
This is a 2x2 table.
D+ = diseased
D- = healthy
T+ = how many test positive
T- = how many test negative
- a = True Positive
- b = False Positive
- c = False Negative
- d = True Negative
What is accuracy?
How many are correctly classified? In this case, how many are true positives and true negatives out of the animals you tested?
What is the difference between apparent prevalence vs. true prevalence?
Apparent prevalence vs True prevalence
b/c tests aren’t perfect, if you now sen and spe of this test you can correct the apparent prevalence, aka the prevalence that you see by applying a test/what it seems to be based on the test you ran, with what the real prevalence in the real world.
- Do not need to know equation.
- Estimating disease prevalence with an imperfect test = apparent prevalence
- Unknown status for each animal
- Test is Positive… is it a true positive or a false positive?
- Test is Negative… is it a true negative or a false negative?
- Using Se/Sp we can estimate True Prevalence from Apparent Prevalence
Clinical Case:
‘Lou’ is a 5-year old Catahoula Leopard intact male dog from
Louisiana. Lou tested for Dirofilaria immitis (Heartworm) using a SNAP test (lateral flow immunoassay, against antigen).
How confident are you that Lou needs treatment?
Calculate the results
Does this information help us?
What is this question asking?
What is the probability that a person is female given they are a veterinary student?
What is this question asking?
What is the probability that a person is a vet student given they are female?
In the U.S, look at all the women in the U.S. and see how many are vet students. A few thousand out of 400 million total people in U.S., half of which are female
What is this concept?
The probability of A given B is absolutely not the same as the probability of B given A.
What do you care about as a clinician?
Positive predictive value.
I care about what is the probability that the animal is diseased given that they test positive.
What do you care about as a clinician?
Negative predictive value.
What is the probability that my animal is healthy given that I have a negative result.
Can we simply use the study numbers to calculate Predictive
Values?
Predicative values are completely prevalence dependent, and therefore you can not use the study values for your patient b/c it depends on the prevalence of that disease in your region.
How does predicative ability changes with prevalence?
Had 240 dogs in study, cut them up, and found 208 had HW, 32 did not have HW = 86% prevalence.
Same sens and spec in study, your + prerdic value = 99% meaning that from just the study alone if you are to ask what is the probab that I will have disease iven a test out of 240 animals, you wokd be 99% correct. But if you have a neg test result out of 204, you are only 50% sure that it does not have HW.
If switch prevalence, but keep sens and spec, and you now have a 5% prevalence –> 12/240 = 5%
Calculated what #s would be based on sens.
PPV went down because predicative value went down.
Set 1000 as # of animals in population
Calculate the PPV or NPV
What is this test’s utility?
Most circumstances
Gold standard
Worry about cost
How can you administer this in a large population?
What is this test’s utility?
For screening or confirming disease status, but not both. If Sp is weak, do not use if disease is very rare.
Sen > 99 = good for screening
if sp is good = good for confirmation
What is this test’s utility?
For circumstances that don’t require high degree of certainty… low consequence to misdiagnosis. 10% of there being an issue.
E.g. with HW, if you misdiagnose you can do more harm than good (if HW burden is too high, you can induce further damage)
What is this test’s utility?
Use with caution
What is this test’s utility?
Why are you even bothering?
What are the classic ways of predictive values?
pre-test odds of disease is also prevalence
Simply just explained the formulas
See below
What is the difference between prior probability and posterior probability?
Test + in red
Test - = in blue
What are clinical decision thresholds?
When you first see a patient, know nothing about it. Then throughout PE, you start to collect information to help you determine whether or not there is disease/problem present.
Run test and it is -, so you are a little off the fence but you are not going to entirely rule out disease.
Run test and it is +, you are now leaning more towards disease but you are not 100% sure.
Depending on how good of a test you use, that will influence your decision making.
LR of 2 = some evidence, think 10% prevalence. Post clinical exam, there is now a 20% probability. Do x ray and this test has strong evidence because LR = 10, now at 70% probability that there is actually a disease in this animal. Do another DG that is a confirmation test so now you are 98% confident you are dealing with this disease. Main takeaway: you are building evidence. Gold standard you can be more 100 % confident.
What is the diagnostic process?
