Measures of Reliablity and Validity

Question 1

requires constant collection, evaluation, analysis, and use of quantitative and qualitative data.

Answer 1

Clinical Medicine

Question 2

Error

Answer 2

• Mistakes in the diagnosis and treatment of patients
• Mistakes due to clear negligence

Question 3

Goal

Answer 3

minimize error in data so as to guide, not mislead

Question 4

PROMOTING

Answer 4

Accuracy and Precision

Question 5

What errors do we need to reduce

Answer 5

Differential and Indifferential Erros

Question 6

We need to reduce what variability?

Answer 6

intraobserver and interobserver

Question 7

closer to the true value

Answer 7

Accuracy

Question 8

Also known as “reproducibility” or “reliability”

•Ability of a test to give the same result or a similar result with repeated measurement of the same factor

Answer 8

Precision

Question 9

Differential error

Answer 9

information errors differ between groups

Question 10

information is incorrect, but is the same across groups

Answer 10

Nondifferential error

Question 11

refers to any systematic error that may occur during the collection of baseline or follow-up data

Answer 11

Measurement bias

Question 12

Examples of Measurement Bias

Answer 12

• Blood pressure values
• measuring height with shoes on
• laboratories and the use of different methods

Question 13

• Variability and unpredictability
• results in lack of precision
• some observations are too high and some are too low

Answer 13

Random error

Question 14

one observer examining the same results more than once

Answer 14

Intraobserver variability (within the observer)

Question 15

Interobserver variability (between observers)

Answer 15

2 or more observers examining the same material

Question 16

measure of the consistency of a metric or a method

Answer 16

Reliability

Question 17

• Overall percent agreement
• Paired observation
• Multiple variables
• Kappa test ratio

Answer 17

MEASURES OF RELIABILITY

Question 18

Common way to measure agreement

Answer 18

Overall Percent Agreement (OPA)

Question 19

• does not include prevalence
• does not show how disagreement occurred
• Agreement might be due to chance alone

Answer 19

Drawbacks of OPA

Question 20

Percent Agreement Formula

Answer 20

PA = a+d/a+b+c+d(100)

Question 21

Measures the extent to which agreement exceeds that expected by chance

Answer 21

KAPPA TEST RATIO

Question 22

Kappa =
(Percent Agreement Observed)-(Percent agreement expected by chance alone)/100% -(Percent agreement expected by chance alone)

Answer 22

FORMULA FOR KAPPA TEST RATIO

Question 23

INTERPRETATION OF KAPPA

Answer 23

<0 = Less than Chance agreement
0.01-0.20 = Slight
0.21-0.40 = Fair
0.41-0.60 = Moderate
0.61-0.80 = Substantial
0.81-0.99 = Almost perfect agreement

Question 24

Ability of a test to distinguish between
WHO HAS a disease and WHO DOES NOT

Answer 24

Validity

Question 25

Screening tests

Answer 25

is performed as a preventative measure

Question 26

- able to correctly identify who has the disease - Reliably finding a disease when it is present - Avoids false negative results

Answer 26

Sensitivity

Question 27

Specificity

Answer 27

– correctly identifies who does not have the disease - Reliably excluding a disease when it is absent - Avoids false positive results

Question 28

Type 1 error / false-positive error / alpha error

Answer 28

Finding a positive result in a patient in whom disease is absent

Question 29

Finding a negative result in a patient whom disease is present

Answer 29

Type II error / false-negative error / beta error

Question 30

Sensitivity Formula

Answer 30

TP/TP+FN

Question 31

Specificity Formula

Answer 31

TN/TN+FP

Question 32

False Positive error rate formula

Answer 32

b/(b+d)

Question 33

False Negative Error rate formula

Answer 33

c/(a+c)

Question 34

Predictive Values

Answer 34

Describes the probability of having actual disease given the results of a test

Question 35

Positive predictive value (PPV)

Answer 35

Indicates what proportion of the subjects with positive test results actually have the disease

Question 36

Negative predictive value (NPV)

Answer 36

Indicates what proportion of the subjects with negative test results actually do not have the disease

Question 37

Formula for PPV

Answer 37

PPV = number of people with gold-standard evidence of disease who test positive (a)/ number of people who test positive (a+b)

Question 38

NPV FORMULA

Answer 38

NPV = number of people with gold-standard absence of disease who test negative (d)/ number of people who test negative (c+d)

Question 39

Sensitivity & specificity versus predictive value

Answer 39

Sensitivity and specificity are characteristics of a test. Positive predictive value (PPV) and negative predictive value (NPV) are best thought of as the clinical relevance of a test.