Appraisal stuff in EBM

Question 1

Give examples of POEM vs. DOE

Answer 1

POEM: morbidity, mortality, quality of life, etc.
DOE: blood pressures, cholesterol level, diagnostic testing results

Question 2

In a therapy study, randomization is

Answer 2

best way to allocate subjects in study to the comparison groups; random number table or computer-generated randomization are best ways (need that person to say in that group throughout the study)

Question 3

In therapy study, allocation concealment is

Answer 3

to ensure that person recruiting for the study does not know the group into which the subject will be placed (worry about unconscious bias); randomization happens AFTER getting the appropriate pool of patients

Question 4

Blinding can be done with

Answer 4

a placebo (e.g. pill for patients to make sure they feel like they’re actually getting a treatment)

Question 5

What are two consequences of having subjects drop out of study?

Answer 5

1. Withdrawals and dropouts might be unequal between groups (would undo equalization effect of randomization)
2. Dropouts might reduce overall power of study: if greater than 20% of the subjects are lost in the study, this could be fatal!!

Question 6

ITT, power, and adjustment for baseline differences mean what?

Answer 6

ITT: subjects must be analyzed in the groups to which they are randomized, if you move subjects from intervention to control group for whatever reason, you’ve broken randomization
Power: 1 - beta, usually set at .8 or .9; alpha is .05 or .01
Adjustment for baseline differences: there may be differences in the groups in important subject characteristics that are MERELY A RESULT OF RANDOMIZATION

Question 7

Questions to consider at the end of your therapy critical appraisal?

Answer 7

Is my patient similar to the ones included in the study?
Is the outcome one that your patients would care about? (“patient-oriented evidence”)
Is the intervention feasible in your practice?

Question 8

Validity of a diagnostic test study (___ and ____ studies) can be assessed by what primarily?

Answer 8

Cohort, case control;
Was there an independent, blind comparison to a recognized reference standard for the condition in an appropriate spectrum of patients?

Question 9

In diagnostic test study, who should get the test of interest and the reference standard? What does independent, blind, and appropriate spectrum of patients mean?

Answer 9

Subjects should get BOTH;
Independent: subjects should get both of the tests; If the researchers only perform the reference standard when the test of interest is positive, then that’s not an independent comparison (e.g. the researchers may want to reserve an invasive reference standard (a procedure or surgery) for only those patients with a positive test of interest, but this still compromises the validity of the study)
Blind: researchers performing test of interest shouldn’t know results of the standard and vice versa
AS of P: patients with range of severity and progression of disease

Question 10

Give an example of an exploratory cohort vs. a validation cohort

Answer 10

for the UTI example - the researchers did urine dipsticks on everyone suspected of UTI, and compared them with urine culture results. they may find in the exploratory cohort that the combination of nitrites, blood and WBC count > 5/hpf best predict UTI. They should then set up a validation cohort study, where this combination is specifically tested (independently, blind, etc.) to diagnose UTI.

Question 11

For applicability, aka generalizability or external validity, what are you asking?

Answer 11

is the test available now?
is it an acceptable test to the patient?
an invasive test is not that useful - it will carry risks that need to be weighed against the risk of the disease you’re trying to diagnose
will I change my management based on the results of the test?
this is the classic question. Using likelihood ratios, if the likelihood of the disease doesn’t change after you’ve applied the test, then there’s really no reason to subject the patient to the test.

Question 12

For systematic reviews, what is your validity criteria?

Answer 12

1. Was the SEARCH for evidence comprehensive? (at least 2 different databases, search for unpublished lit, likely relevant studies were missed?)
2. Were there explicit INCLUSION and EXCLUSION criteria stated and followed? (included table of excluded studies?)
3. Were the included studies appraised for their validity?
4. Was there an assessment for heterogeneity? Heterogeneity can be about differences due to patient population, methods, etc.; also p-values and I-squared
5. Did the authors assess for publication bias? Was a funnel plot provided?

Question 13

In a SR, meta-analysis can be performed on both ___ and ___ data?

Answer 13

continuous; dichotomous;

weighted mean difference for continuous data or a relative risk for dichotomous data, e.g.

Question 14

For SR applicability, three questions to consider?

Answer 14

1. Is my patient like the patients in the review (look at inclusion and exclusion criteria)?
2. Is the intervention feasible in my practice?
3. Is the outcome one my patients would care about (patient-oriented evidence)?