2.4 Application of statistics Flashcards
Define uncertainty and measurement of uncertainty (MOU)
Uncertainty- Doubt about accuracy of a measurement
MOU- quantitative estimate of variability of dispersion of results about a single measurement if it was
to be repeated at another time
total error- usually if traceable standard then bias is eliminated but if not then imprecision AND bias contribute
15189- trueness derived from participation in EQAP (has its own issues)
How do you estimate MOU?
- Define what will be measured
- Identify all sources of uncertainty- hard to do- and quantify uncertainties of each of these sources for combined uncertainty
in practice CV% x2 obtained from internal control - at least 30 sets of data over period of time- often used as combined uncertainty
(other source is EQAP CV% method, or kit insert)
there is a 95% chance that true result lies within range covered by test result +/- MOU
2.77x cv of the method (NPAAC 2007)
2.77x 20%= 55%
two results for dsDNA at level of approx. 10IU/mL need to differ by >55% for there to be 95% confidence that they are in fact different
What are some of the sources of “uncertainty” in measurement?
All points in the request-test-report cycle
1. pre analytical : patient prep, specimen collection, transportation, storage, prep of primary sample , patient specific (jaundice, drugs, heterophile)
- analytical: uncertainty of calib value and dispensed volume, reagent/calib batch, equipment aging/maintenance, operator, environmental fluctuations , components of measurand itself (epitope)
- post analytical: transcription/LIS interface, uncertainty in result interpretation
What is the coefficient of variation (CV)? And what are some of the considerations when calculating CV?
CV is measure of spread that describes amount of variability relative to the mean - precision marker
CV = (SD/mean)x 100
CV needs to be at a certain level/cut off
Esp important around the clinical decision point (internal QC)
e.g.
dsDNA at levels between 6-12IU/mL (>7IU/mL pos)
Over 65 runs QC mean value 10IU/mL, CV of 20%
As number of replicates increased, standard error is reduced
Name some performance characteristics of an assay that is important
- analytical sensitivity: rate of change of response with change in analyte conc, ability to distinguish small changes
- selectivity/interference: accuracy in measurement in presence of interfering components in sample matrix (not specificity)
- clinical utility
– sensitivity: correctly identify true positive rate
– specificity: correctly identify true negative
–pos/neg predictive value: dep prevalence
PPV- if test result pos how well predict having disease,
NPV- if test neg, how well predict not having disease - precision: closeness of agreement between serial measurements from sample sample (cv, sd)
- intra assay/repeatability, inter assay/reproducibility (variable conditions)
- accuracy/trueness: if lacking then either random or systemic error, expressed as bias % deviation from reference value (need reference method)
- 15% acc reference value
- 20% near the LLOQ (lowest conc that can be measured with suitable precision and accuracy)
Define what is a measuring range?
The range between LOQ (the lowest conc that can be measured with suitable precision CV<20% and accuracy) and the upper limit of linear calibration
Should be appropriate to analyte conc likely to be encountered in samples
How many samples are required to adopt a reference range?
Well depends
- if adequate RR study and adopting then may be ok/verification for 20 samples +/-20 for transfer
- from comparative reference method 40 samples
- establishing new RR requires minimum of 120 samples
What is the acceptable precision?
What is inter and intra assay CV?
intra assay- repeatability
At least 3 replicates at 3 conc (20x run on same run)
<10% acceptable
inter assay- 3 conc over 10days/ usually different concentrations
<15% acceptable
<20% if near lower limit of quantitation
Needs to be assessed near extremes of calib range and in middle (also near cut off)
How does sensitivity and specificity interact for a rare condition?
PPV/NPV are affected by prevalence while as sens/spec is not. Therefore it depends on the testing population.
If rare disease, maximize pre test probability to help improve PPV/NPV
Institute step path review to remove risk of false positive- high risk- regardless of high specificity test is if the disease is rare
PPV value is low if pre test prob low, regardless of test characteristics
keep a two tier test system- ppv
What is the difference between precision and accuracy?
Precision is dispersion around the mean , agreement with each other
reproducibility- between runs
repeatability- within runs
Deterioration of reagents, instability of machine or operator problems (quality control)- fix
Random errors
Accuracy- is the mean of the sample where it should be? closeness of measurement to its true actual value, ability to measure analyte correctly on any given occasion
calibration - fix
Systemic error issues
Lower limit of normal IgG4 0.04g/L what are concerns around results of this range?
Very close to zero
At low concentration poor precision, CV high, cant expect precise and reproducible results
Requires careful interpretation
What steps do you take if you want to transfer a RR from an external source? Name two sources of ext RR
Source: manufacturer, reference paper
Transfer: Samples from a reference population - need exclusion criteria, partioning criteria (Age/gender) with consent
Pre analytical esp patient prep, collection, storage analytical and population must be comparable
At least 20 samples collected to verify manufacturers RR
- if <2 out of 20 fall outside suggested reference can be adopted
- if 3 or more then need to run 20 more samples
What steps do you need to take for a lab to establish own RR?
Need a reference population of at least 120 samples, exclusion/partitioning criteria - include healthy controls, disease controls
Control for
- pre analytical
- analytical factors;Test with same method on same instrument
Plot data on histogram- deal with outliers
Perform normality test (P-P, QQ plot against theoretical normal distribution)
Results will be Guassian/parametric or non Guassian
if non parametric, try and transform the data (log or square root transformation, others like box cox etc.)
Gaussian- 95th percentile determined as mean +/-2SD
Non Gaussian/skewed- rank results ascending and then 0.025x(n+1) and 0.975x(n+1) n= sample size
What do reference values do?
RR allow the comparison of observed/result data to reference data from a defined population
Usually defined population is healthy
Which data transformation strategies should be employed for neg or pos skewness?
Neg skewness (left)- take squares, logarithmic
Pos skewness (right)- logarithmic
