(Reliability and validity)

Question 1

Quality assurance

Answer 1

Prior to data collection (through protocol and manuals of operation)

Question 2

Quality control

Answer 2

During data collection and processing

Question 3

How do we measure reliability?

Answer 3

• % agreement
• kappa
• icc
• bland-altman plots

Question 4

How do we measure validity?

Answer 4

• sensitivity/specificity
• ROC/c-statistic
• Bland-altman plots

Question 5

How to adjust for self-reporting bias

Answer 5

with a validation study and by directly adjusting for mismeasured exposures using the relationship bw the gold standard measurement and mismeasured exposure

Question 6

Sensitivity=

Answer 6

True positives/(all positives)

Question 7

Specificity=

Answer 7

True negatives/(all negatives)

Question 8

Positive likelihood ratio

Answer 8

Sensitivity/(1-specificity)

Question 9

Negative likelihood ratio

Answer 9

(1-sensitivity)/specificity

Question 10

Limits to extrapolating Se and Sp across studies

Answer 10

• Se and Sp for specific tests often not directly generalizable across populations
• Depending on severity of condition, prevalence of other diseases that would also test positive, association between self-report and other characteristics

Question 11

Which of Se and Sp is important to determine the bias in the effect measure

Answer 11

Specificity

Question 12

ROC curve: what happens when we select a higher criterion value (axis x)

Answer 12

false positive fraction decrease with increased specificity
but
true positive fraction and sensitivity will decrease

Question 13

Uninformative test has AUC of…

Answer 13

0.5 (area under the curve)