Critical appraisal

Question 1

what is selection bias?

Answer 1

a systematic difference in the recruitment of comparisson groups attributable to incomplete randomisation

…a few really sick patients were given the intervention on compassionate grounds

Question 2

what is performance bias in an RCT?

Answer 2

a systemiatic difference in the treatment being provided to each comparison group, apart from the intervention being studied

Question 3

what is exclusion bias in RCTs?

Answer 3

systematic difference in the number of patients who withdraw/do not complete follow-up between the comparison groups

Question 4

what is detection bias in RCTs?

Answer 4

a systematic difference in the methods used for measuring outcomes between the comparison groups

Question 5

what is the problem with a non-randomised controlled clinical trail?

Answer 5

non-randomisation avoids controlling selection bias.

this usually means there will be a distinct difference between the intervention and control group that invalidates any difference measured being attributable to the intervention

Question 6

what 4 questions should you ask about a study population?

Answer 6

• How were the subjects receruited?
  
  • err: advertising in the newspaper rather than asking service users
• Who was included in this trail?
  
  • err: pharmakokinetic trials in 23-year-old male volunteers
• Who was excluded from this trial?
  
  • err: moderate-/severe-heart failure patients from hospital outpatients, excluding mild disease from general practice
• Were these patients treated in ‘real-life’ circumstances?
  
  • err: admitted to hospital to monitor compliance

Question 7

what is recruitment bias?

Answer 7

a systematic error in identifying the group of volunteers from which the samples are selected

…advertising your study in the newspaper will have a recruitment bias towards people motivated to read the newspaper

Question 8

what biases are most common in prospective cohort studies?

Answer 8

1. selection bias
   
   1. are the cohorts selected really from the same population?
   2. are any group of subjects at particular risk of developing an outcome?
2. attrition bias
   
   1. >20% loss-to-follow up of either group is too much
   2. systematic difference in withdrawal in either cohort
   3. systematic difference in those who withdrew and those that stayed within cohorts

Question 9

what biases are most common in retrospective cohort studies?

Answer 9

1. selection bias
2. inadequate standardisation of outcomes measurment
   
   1. will be taking data from the medical records of the patients in each cohort. the investigators will have no way of controlling how these data were collected
   2. maybe incomplete/inaccurate. the dismissial of these patients from the cohort introduces a resulting attrition bias

Question 10

what is a case-control study?

Answer 10

a retrospective observational study used to identify possible causative factors contributing to the disease

identify a cohort of ‘cases’ and a matching cohort of ‘controls’ without the disease and look back through records to identify a common exposure in the case cohort that the outcome could be attributable to

Question 11

what is the name of the critical appraisal checklist available for all study deisgns?

Answer 11

CASP - critical appraisal skills programme checklist

Question 12

what is the STROBE statement?

Answer 12

a checklist of items that should be included in any observational studies

Question 13

what is the CONSORT statement?

Answer 13

a 25-item checklist and flow chart used to increase the completeness and transparency of reporting clinical trials, pub 2010

Question 14

what is the PRISMA statement?

Answer 14

this is the 2015 checklist for the design and reporting of systematic reviews and meta-analyses

should be used by authors when writing their manuscript and by journal editors when deciding what to publish

Question 15

what are the five elements of critial appraisal?

Answer 15

1. what is the research question? (PICO)
2. what is the design of the study?
3. assess the study’s validity (detecing biases)
4. assess the study’s results (statistical and clinical significance)
5. assess the study’s generalisability and applicability

Question 16

explain not statistically significant BUT clinically important

Answer 16

most likely to occur in small sample sizes that are underpowered

you have identified a clinically significant difference between the intervention and control group, but the numbers of the study are too small for the difference to be statistically significant.

Question 17

explain statistically significant BUT NOT clinically important

Answer 17

likely to happen in large sample sizes, where small differences in outcomes that may not actually be important to patients and their perception of the course of disease will be detected as statistically significant

should be contrasted with patient-centred research. make sure that what you are reporting to be important in the numbers has a tangible positive effect to your patients disease course

Question 18

what is the purpose of a non-parametric statistical analysis?

when should a non-parametric test be used?

Answer 18

a non-parametric test does not assume that your test population follows an underlying distribution

1. data not normally distributed
2. data significantly skewed, such that the median > mean to represent the central tendancy of the data
3. non-normal sample size smaller than the central limit theorem (n= 30)
4. significant outliers that cannot be removed

Question 19

what parametric and non-parametric tests should be used for:

comparing two independent samples drawn from the same population

Answer 19

two-sample, unpaired t-test

Mann-Whitney U-test

Question 20

what parametric and non-parametric tests should be used for:

comparing two sets of observations drawn on a single sample in two different measurements

Answer 20

two-sample, paired t-test

Wilcoxon match-pairs test

Question 21

what parametric and non-parametric tests should be used for:

comparing three or more observations made on the same sample

Answer 21

ANOVA (one-way analysis of variance usuing total sum of squares)

Analysis of variance by ranks (Kruskall-Wallis test)

Question 22

what parametric and non-parametric tests should be used for:

assessing the strength of straight-line association between two continuous variables

Answer 22

Pearson’s (r) product-moment correlation coeffifienct

Spearman’s rank correlation coefficient (rho)

Question 23

what is a Chi^(2) test used for?

Answer 23

testing the null hypothesis that the proportions of variables estimated from two (or more) independent samples are the same

Question 24

what is standard deviation?

Answer 24

SD is the sqrt(varience) , and is a standard measure of the variance of the data set

i.e. a representation of how far the sample points differ from the sample mean

Question 25

what is the standard error of the mean?

Answer 25

SEM is a measure of how far the sample mean deviates from the population mean. it is always smaller than SD

SEM = SD/sqrt(n)

Question 26

what is the difference between 95% CIs and SEM in reporting?

Answer 26

95% CI’s - the population mean will be contained within this range 95% of the time
If the error bars between two populations do not overlap then they may be statistically significant, but you should still check that the correct statistical test was done.

SEM - the population mean will be contained within this range approx 68% of the time, and within 2x(range) 95.4% of the time.
you cannot decide if two populations are different because the error bars of SEM do not overlap

Question 27

what is the definition of a p-value?

Answer 27

the probability that the observed effect, or any more extreme, occurs given that the null hypothesis is true

Question 28

what is the relative risk/risk ratio?

Answer 28

the ratio of the probability of the outcome occurring in the exposed group to the the probability of the outcome occurring in the control group

Question 29

what is the definition of odds ratio?

Answer 29

the ratio of the odds of the outcome occurring in the exposed group to the odds of the outcome occurring in the control group

Odds = p/(1-p)

Question 30

what is a clinical susceptability bais?

example?

Answer 30

if one condition (say PMS) predisposes for another (say endometrial cancer) then statistical analysis may reveal a false association between treatment (say estrogens) given for the first condition and developing the second condition

1978 Cancer Research Feinstein & Horowitz

Question 32

what is wrong with the following statement:

‘the P value <0.05, therefore the null hypothesis is false’

Answer 32

the P value is calculated assuming the null hypothesis is true.

it is NOT the probability of the null hypothesis being true or false.