Setting up a molecular service Flashcards
Role of molecular diagnostics in microbiology
What are benefits of setting up a new molecular testing?
automation
large throughput
fast turnaround time
reduce risk of contamination
internal quality control
interface with reporting software
What are drawbacks of setting up a new molecular testing?
capital investment
training
IT interface
high sensitivity - false positive
Setting up a new molecular service
What are the basic steps involved in a proposal?
select new service
physical space
staff/ resources
funding
expected results
expected impact
timeline of events
Setting up a new molecular service
How do you evaluate your service is working?
Audit
quality assurance - internal/ external
Troubleshooting SOP
Feedback - staff/ end users
Setting up a new molecular service
What costs are involved?
Equipment - including consumables
Maintenance
New staff
Training of staff
As a newly appointed Consultant Virologist, you have been asked to support validation
and verification of a quantitative JCV DNA assay.
13a How would you define the term validation?
ISO 15189
Process of confirming that your in house test works as expected. By gathering evidence during testing for accuracy, repeatability, sensitivity, specificity
To document this ability, each laboratory should produce a Validation File for each method or system which has not been validated formally. The file will include a range of information and will have a different emphasis depending on whether the laboratory is using a modification of commercial systems or has developed an in-house system. Typically the file will include sections such as validation data, tests on known samples, workbooks and relevant publications.
Validated test methods or equipment do not need any further validation after it is done once. Ongoing fitness for purpose is monitored through the laboratory’s quality assurance which may include personnel competency assessment, quality control,
proficiency testing, etc. It should also be noted that validation is performed on in-house diagnostic tests to validate sensitivity, specificity, precision and, in case of a quantitative test, linearity
The intention of validation is to provide documented evidence that a diagnostic test is performing within required specifications and fit for purpose.
This may involve results of experiments to determine its accuracy, sensitivity, specificity, reliability, repeatability reproducibility, and uncertainty of measurement. A validation may be extensive, for
example to validate a newly developed in-house method, or narrow in scope, for example to validate a commercial method which is already in use and has had minor modifications
For methods already in use for which no specific existing validation is in place, it is important to provide documentary evidence which supports reasons for their use. It is usually sufficient to prepare a file based on historical evidence, such as results from comparisons or other studies undertaken, copies of published papers, EQA, IQA and IQC results etc. Work book records can be cross referenced if appropriate in the validation report.
As a newly appointed Consultant Virologist, you have been asked to support validation
and verification of a quantitative JCV DNA assay.
13b How would you define the term verification?
ISO 15189: 2012 defines “verification as the confirmation, through provision of
objective evidence that specified requirements have been fulfilled”. It can also be
described as the confirmation of whether or not a product (for example commercial kit
system or equipment) complies with a regulation, requirement, specification, or
imposed condition.
Verification should be performed in scenarios such as when a laboratory wants to
introduce a new validated commercial method/equipment with defined performance
(from manufacturer) for routine use or in a case where a previously validated method
is modified and then revalidated before use.
The laboratory verifies their ability to achieve acceptable results with the method or system in question. It should be noted that verification is done through performance parameters and it varies considerably with the type of test method/equipment being verified. It is an ongoing process.
The purpose of verification is to confirm whether or not a product (for example a
validated method, commercial kit system or equipment) is complying with a regulation,
requirement, specification, or imposed condition. The minimum tested attributes in
verification may include accuracy, precision and linearity but are not limited to these.
Refer to Appendix 4 for a summary of what a verification report should contain
As a newly appointed Consultant Virologist, you have been asked to support validation
and verification of a quantitative JCV DNA assay.
13b Your lab buys an off-the-shelf kit from a manufacturer. What validation and verification steps need to take place?
Manufacturer should validate the kits.
User should verify the kits.
Validation of commercial assay kits as well as the equipment used should be
performed by the manufacturer to ensure that they achieve the stated performance.
The user should obtain this information from the manufacturer. However, verification
should be performed by the user, confirming through review (published and
unpublished evaluations, EQA data, etc.) and testing that the equipment and the
commercial kits meet the written specification requirements1,6
As a newly appointed Consultant Virologist, you have been asked to support validation
and verification of a quantitative JCV DNA assay.
13c It is anticipated that you perform some testing using the new method and compare
these results against an established standard test.
Please list at least 4 controls for your validation [2 marks]
List potential sources (including commercial sources)
negative control
negative patient sample
positive standard test control
positive new test control
positive patient sample
low positive patient sample
quantitative assay so there is benefit in using different strength patient samples
Sources:
old patient samples/ IHC
manufacturer new IHC/ new standard test
external controls from collaborating laboratory
external quality assurance
off-the-shelf control
List 8 assay performance characteristics you should consider during this verification
and/or validation of diagnostic methods?
sensitivity - ability to detect true positives
specificity - ability to detect true negatives
PPV - when result is positive, it means patient has virus
NPV - when result is negative, it means patient does not have virus
repeatability - closeness of results if repeated twice
reliability - ability to maintain performance without breaking down
accuracy - the closeness of agreement between the value obtained from a large
series of test results and an accepted reference value. Accuracy refers to how close a measurement is to the true or accepted value.
precision - Precision refers to how close measurements of the same item are to each other
measurement of uncertainty - results which are very low pos which could be attributed to equipment/ procedure artefact
linearity - how results change due to concentration. e.g 1000 copies will be 10x strogner signal than 100 copies. This should be a straight line
User feedback - acceptability
Safety assessment - COSHH risk assessment
What is the difference between precision and accuracy?
Precision and accuracy are two ways that scientists think about error.
Accuracy refers to how close a measurement is to the true or accepted value. Precision refers to how close measurements of the same item are to each other.
Precision is independent of accuracy. That means it is possible to be very precise but not very accurate, and it is also possible to be accurate without being precise.
The best quality scientific observations are both accurate and precise.
Dart board analogy
