Critical appraisal course Flashcards

1
Q

What is EBM

A

The conscientious, explicit and judicious use of current best evidence in making decisions about the care of a patient

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

5 steps of EBM

A
  1. Clinical question
  2. EVidence
  3. Critical appraisal
  4. Application
  5. Implementation and monitoring
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Stages of critical appraisal

A
  1. The clinical question
  2. Methodology: study design, recruitment, variables and outcomes
  3. Results: data analysed and differences between groups assessed for significance
  4. Applicability
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Internal and external validity

A

Internal: extent to which the results from the study reflect the true results
External: extent to which study results can be generalised

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Efficacy and effectiveness

A

Efficacy is the impact of interventions under optimal (research) setting

Effectiveness is whether the interventions have the intended or expected effect under ordinary clinical settings

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

The efficacy of an intervention is almost always better than the effectiveness

A

True

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

The acceptance of EBM means all clinicians practice the same

A

False

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Patient values have no role to play in EBM

A

False

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

The effectiveness is almost always better than the efficacy

A

False

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

A research project taking place in the outpatient clinic is almost always going to give effectiveness data rather than efficacy data

A

True

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

An RCT can answer any type of clinical question

A

False

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

The clinical question determines which study designs are suitable

A

True

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

A clinical question can usually only be answered by one type of study design

A

False

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

The critical appraisal should start by examining the study design

A

False

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Broad categories of studies and what they can achieve

A
  1. Observational descriptive
    - survey, qualitative, case report/series
    - Generate hypothesis
  2. Observational analytical
    - case-control (outcome->exposure)
    - cohort (exposure->outcome)
    - test hypothesis
  3. Experimental
    - RCT, crossover, N of 1
    - intervention
  4. Others
    - Ecological: information about the population
    - Pragmatic: real life environment. Example- all people in clinical location, outpatient, randomised to receive particular treatment. More reflective of everyday practice
    - Economic
    - Systematic review/MA
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Case series is observational analytic

A

False

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Case control is observational descriptive

A

False

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Qualitative study

A

Opinions are elicited from a group of people with emphasis on subjective meaning and experience. Complex issues can be identified

Data gathering and data analysis develops iteratively-> results inform further samplig

Inductive-> knowledge generated through data sampling and gathering as opposed to other scientific methods where research is deductive

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Other names for case control

A

Retrospective or case comparison

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

Case control- type, advantages and disadvantages

A

People with outcome variable, are compared to those without outcome variable, to determine risk factors they have been exposed to in the past

Adv: 
cheap and easy
good for rare outcomes
few subjects required
good for diseases with long duration between exposure and outcome

Dis:
not for rare exposures
Recall problems
Control groups can be difficult to select

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Cohort study- design, retrospective type, other names, adv, disadv

A

A group of people with exposure are followed up to see the development of an outcome

Also called prospective or follow-up
Retrospective type is using cohort data that already exists (say from 20 years ago, with exposure)

Adv:
Good for rare exposures
multiple outcomes
temporal relationship
estimation of outcome incidence rates
Dis:
May take ++time from exposure to outcome
expensive
attrition rates
unsuitable for rare outcomes
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

Recall bias is a bigger issue fir CC or Cohort

A

Case control

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

Which is better study design for rare exposures

A

Cohort

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

Whch study design is betten when long time from exposure to outcome

A

CC

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Disadvantages of RCT
Expensive | Time consuming
26
When might you use a crossover trial
When unable to get enough subjects for RCT | Given one intervention, then switched half way through
27
Disadvantages of cross-over trials
``` The order f interventions might be important Carryover effects (drugs with long half lives) or prolonged discontinuation/withdrawal May be difficulty using historical controls if conditions were different ```
28
N of 1 trials
Experimental version of case report Single person Given randomised treatments Report on response
29
Historical control bias is an issue in which type of study design
Crossover
30
Which types of biases are an issue in open label
Selection and observation bias
31
Two sources of methodological error
Bias | Confounding factors
32
Definition of bias
Any process at any stage of inference, which tends to produce results or conclusions that differ (systematically) from the truth Not by chance Researchers must try to reduce bias
33
categories of bias
1. Selection bias= recruitment of sample 2. Performance bias= running of the trial 3. Observation bias= data collection 4. Attrition bias 2+3= measurement bias
34
Bias versus confounding
Confounders are real life relationships between variables that already exist, and so are not introduced by the researcher
35
Selection bias
Error in recruitment of sample population ``` Introduced by: Researchers= sampling bias - admission or berkson - diagnostic purity - neyman bias - membership bias - historical control Subjects= response bias - volunteers differ in some way from the population - example if more motivated to improve their health, and adhere ++to the trial ```
36
Types of sampling bias
1. Berkson bias - sample taken from hospital setting, and therefore rates/severity of condition is different compared to target population 2. Diagnostic purity bias - co-morbidity are excluded from sample population, and therefore does not reflect the complexity in the target population 3. Neyman bias - prevalence of condition, does not reflect the incidence, due to time gap between exposure and actual selection. such that some with exposure are not selected (have died) - example, giving treatment following MI. Some may die shortly after MI and therefore not be selected, therefore there is better prognosis already 4. Membership bias - members of group selected may not be representative of target population 5. Historical control bias - subjects and controls chosen across time, so definitions, exposures, diseases and treatments may mean they cannot be compared to one another
37
Protecting against performance bias
Systematic differences in the care provided, apart from intervention evaluated Standardisation of care protocol and blinding protects Randomisation and blinding
38
Types of observation bias
Failure to measure or classify the exposure or disease correctly. Can be due to researcher or participant. Researcher 1. Interviewer (ascertainment) bias - when researcher not blinded, may approach subject differently depending on if they know they are taking the treatment or the placebo 2. Diagnostic/exposure suspicion bias 3. Implicit review bias 4. Outcome measurement bias 5. Halo effect- knowledge of patient characteristics influences the impression of patients with respect to other aspects Subject 1. Recall bias 2. Response bias- answers questions in a way they think the researcher would want 3. Hawthorne effect- behaving in a way, usually positively, as aware being studied 4. Social desirability bias 5. Bias to middle and extremes 6. Treatment unmasking
39
Attrition bias
The numbers of individuals dropping out differs significantly between the groups Those left may not reflect the sample or target population Intention to treat analysis will need to be conducted
40
Bias occurs by chance
F
41
lack of blinding could lead to ascertainment bias
T
42
What is a confounder, positive and negative
When there is a relationship between two variables, that is attributable to, or confounded by the presence of a third. May make it seem like the two variables are associated when they are not (positive confounder eg coffee and lung cancer *smoking, overestimating association), or fail to show association when there is (negative confounder poor diet and CVD *exercise, underestimating association) To be a confounder 1. Must be associated with the exposure, but not the consequence 2. The outcome, independently from the exposure
43
Controlling for confounders
1. Restriction - inclusion and exclusion criteria 2. Matching 3. Randomisation
44
Accounting for confounders using statistical methods
1. Stratified analyses - can only control for a few 2. Multivariable analysis - Accounts for many, need at least 10 subjects, in a logistic regression If matching was done, McNemar test or conditional logistic regression
45
Simple randomisation
Subjects are randomised to groups as they enter the trial | Selected independently of each other
46
Block randomisation
Differs from simple randomisation in that subjects are not allocated independently Subjects are assigned to "blocks", which as they fill, then distributed evenly to intervention or control group
47
Stratified randomisation
Subgroups are formed in relation to a confounding factor, then in each stratum, block randomisation occurs, so the confounders are equally distributed
48
Concealed allocation and methods
When the treatments being administered in the different arms of the study remain secret Part of the randomisation process Methods: 1. Centrally controlled 2. Pharmacy concealed 3. Sequentially numbered, opaque, sealed envelops 4. Numbered/coded bottles or containers
49
Allocating an equal number of subjects in each groups is possible with simple randomisation
Yes- possible by chance
50
Pygmalion effect (Rosenthal effect)
Subjects perform better than others because they are expected to Power of positive expectations George Bernard Shaw- Pygmalion
51
Placebo effect
In healthcare, the pygmalion effect is often called the placebo effect
52
Latin meaning of placebo
"I shall please"
53
2 methods to make expectations equal
1. Allocation concealment - at time of selection 2. Blinding - once the subjects start treatment
54
Problem with single blinding
If the subject is blind, the researcher is still in full possession of the facts The subject may still be influenced by the behaviour of the researcher who may have expectation about the outcome
55
Blind assessment
Assessment of the outcome measures during and at the end of the study is made without any knowledge of what the treatment groups are
56
Placebo factors
Multiple pills Large pills Capsules
57
Reliability definition and subtypes
Consistency of results on repeat measurements by one or more raters over time 1. Inter-rater - level of agreement by 2+ assessors at the same time 2. Intra-rater - one rater, same material, different time 3. Test-retest - level of agreement from initial test results to repeat measures at later date 4. Alternate form reliability - reliability of similar forms of the test 5. Split-half reliability - reliability of test divided in two, with each half being used to assess the same material under similar circumstances
58
Quantifying reliability
Compare the proportion of scores which agree, with the proportion that would be expected to agree by chance Reliability co-efficient
59
Kappa (Cohen's) statistic k
Kappa or Cohens is used to test measures of categorical variables Inter rater reliability in qualitative Also known as chance-corrected proportional agreement statistic Measures the proportion of agreement over and above that expected by chance If agreement is no more than expected by chance k=0 To be significant, k> 0.7 is normally necessary
60
Strength of agreement or association
``` 0= chance agreement only <0.2 poor agreement beyond chance 0.21-0.4 Fair agreement beyond chance 0.41-0.6 Mod 0.61-0.8 Good agreement 0.81-1.0 Very good 1 Perfect agreement ```
61
Crohnbach's alpha is
Used in complicated tests with several parts measuring several variables When you have multiple Likert questions Internal consistency/reliability No formal test statistic >0.5 mod >0.8 excellent
62
Intraclass coefficienct
Used for tests measuring quantitative variables, such as BP
63
Validity and subtypes
Extent to which a test measures what it is supposed to measure 1. Criterion- predictive, concurrent, convergent, discriminant 2. Face 3, Content 4. Construct 5. Incremental
64
Criterion validity
demonstrates the accuracy of a measure or procedure by comparing it with another measure or procedure that has been demonstrated to be valid 1. Predictive: extent to which the test can predict what it theoretically should be able to predict 2. Concurrent validity: extent to which the test can distinguish between two groups it theoretically should be able to distinguish 3. Convergent validity: the extent to which the test is similar to other tests that it theoretically should be similar to 4. Discriminant validity: the extent to which the test is not similar to other tests that it theoretically should not be similar to.
65
Other types of validity
1. Face validity: superficially looks to measure what it should 2. Content: measures variables that are related to that which should be measured 3. Construct validity: extent to which a test measures a theoretical concept by a specific measuring device or procedure 4. Incremental validity: the extent to which the test provides a significant improvement in addition to the use of another approach. A test has incremental validity if it helps to the use of another approach.
66
Intention to treat analysis
All the subjects are included in the analyses as part of the groups to which they are randomised, regardless of whether they completed the study or not.
67
Last observation carried forward, disadvantages
Way of accounting for subjects that drop out before the end Disadvantages: 1. Underestimation of treatment effects - intervention expected to lead to an improved outcome 2. Over estimation of treatment effects Intervention expected to slow down a progressively worsening condition
68
per protocol analysis
only those subject remaining in the study are used in the analyses introduces bias through exclusion of participants who dropped out
69
when is incidence preferred over prevalence and vice versa
When disease is frequent and short duration-> incidence When long duration, slow, rare-> prevalence more useful to indicate impact of disease on the population
70
Mortality rate
Type of incidence rate that expresses the risk of death in a population over a period of time
71
Standardised mortality rate
Adjusted for confounding factors
72
Standardised mortality ratio
Ratio of observed mortality rate compared to expected mortality rate
73
Types of data summary
``` 1. Categorical Nominal Ordinal 2. Quantitative Discrete Continuous ```
74
Categorical data
No numerical value Not measured on scale No in between values 1. Nominal= unordered - binary, dichotomous= only 2 mutually exclusive categories (dead/alive, male/female) - multi-category= mutually exclusive categories, bearing no relationship to each other (married, engaged, single, divorced) 2. Ordinal= numbered - order inherent, but not quantified - can assume non-parametric
75
Quantitative data (numerical)
1. Discrete - counts (number of children, asthma attacks) 2. Continuous - can have a value within the range of all possible values (age, body weight, ht, temp)
76
Another term for normal distribution
Gaussian distributioni
77
Statistical tests to describe samples with different data samples: categorical, quantitative
Categorical= mode, frequency Quantitative= 1. Non-normal distributed-> median, range 2. Normally distributed-> mean SD
78
Advantages and disadvantages of median
``` Adv: robust to outliers Dis: does not use all data not easy to manipulate mathematically ```
79
Explain interquartile range
Median = 2nd quartile (50%) First quartile is at 25% Third quartile is at 75% So interquartile = 3rd quartile- 1st quartile
80
the midpoint of a perfect ND also represents
Mean value of the concerned parameter in the population
81
SD from mean, values contained, adv and disadv
``` 1 SD = 68 & of values 2 SD (1.96) = 95% 3 SD = 99% (2.?58) ``` SD= calculated as square root of variance Variance is the sum of all differences between all the values and the mean, squared and divided by total number of observations = 1 (degree of freedom) Adv: uses all the data, when distribution normal, mean and SD summarise entire distribution Dis: vulnerable to outliers, not useful for skewed data
82
Standard error
SE of a mean is an estimate of the SD that would be obtained from the means of a large number of samples from that population If we measure ht from sample of population, calculate mean. Get another sample from people of same population, and measure hts of people, calculating another mean. Unlikely to be the same as before. Can carry on recruiting samples and calculating means. Series of means. If we calculate the mean heights for each samply, the plot the frequency of means, end up with ND. Mean is population mean. Spread of observations around population mean is known as SE.
83
Confidence intervals
Tells us the range within which a true magnitude of effect lies with a certain degree of assurance, usually 95% CI for population mean= mean +/- 1.96 x SE (SD/sqRn))
84
Positive and negatively skewed data
Positively skewed has longer tail to the right | Negatively skewed has longer tail to the left
85
Null hypothesis
There is no difference, no association between two or more sets of data Any observed association can occur by chance. The probability of results occuring by chance can be calculated. If unlikely to be due to chance, the null is rejected
86
Alternate hypothesis
Experimental hypothesis
87
Probability
Likelihood of an event occurring as a proportion of the total number of possibilities Expressed as P values
88
P values
Probability of getting the observed results or more extreme, given a true null hypothesis Significant= result deemed unlikely to have occurred by chance, thus rejecting the null <0.05 probability to obtain result by chance is <1 in 20 <0.1 by chance <1 in 10 <0.5 chance 1 in 2 Statistical significance is very sensitive to sample size and study power
89
Comparing statistical significance and clinical signifiicance
Statistical significant tells use whether results are due to chance Clinical significance tells us whether the results are worthwhile or even noticeable
90
One tailed and two tailed significance testing
1-tailed examines in only one direction, ignoring the other | In 2 tails examines both directions
91
Type 1 error
Null hypothesis rejected when in fact true= false positive Usually attributable to bias or confounding Avoided by using stats to generate value P value <0.05 signifies null can be rejected. >0.05 null cannot be rejected, therefore minimising type 1 error Significance level a= pre chosen probability P value = probability of making type 1 error Not affected by sample size More likely with increasing number of tests/end points
92
Type 2
Null hypothesis is accepted when it is in fact false= false negative Usually because sample size not big enough Probability of type 2 error= b B depends on sample size and alpha B gets smaller as sample size gets bigger B gets smaller as the number of tests of end points increases
93
Power
Probability that a type 2 error will NOT be made in that study A power of 0.8 generally accepted= probability of finding a real difference when one truly exists Probability of rejecting the null hypothesis when true difference = 1-B
94
Unpaired and paired data
Unpaired comes from two different groups/subjects Paired data comes from the same subjects at different times
95
To identify type of statistical test to use
1. Consider if descriptive, comparing two groups or comparing >2 groups 2. Then consider if categorical, non-normal or normal 3. Paired or unpaired ``` Therefore: 1. Descriptive- categorical= mode, freq non-normal= median, inter-quartile normal= mean, SD ``` 2. Comparing two groups categorical= chi square for large sample, fisher's exact for small non-normal= Mann-Whitney U (unpaired), Wilcoxon's rank sum (paired) normal= students T, either paired or unpaired 3. Comparing >2 groups categorical= chisq (unpaired), McNemars (paired) Non-normal= Kuskal-Wallis ANOVA (unpaired), Friedman (paired), normal= ANOVA (paired or unpaired)
96
Contingency table
Categorical Row= treatment groups, exposure Columns= outcomes/disease status
97
What would be the appropriate test for comparing 2 groups of unpaired data that is not normally distributed
Mann-Whitney U
98
What would be the appropriate test for comparing 2 groups of paired data that is not normally distributed
Wilcoxon's rank sum test
99
What would be the appropriate test for comparing > 2 groups of paired categorical
McNemar's
100
What would be the appropriate test for comparing more than 2 groups of unpaired data that is not normally distributed
Krushkal willis ANOVA
101
Measuring BP in a subject before and after is an example of paired
Yes
102
What would be the appropriate test for comparing 2 groups of unpaired categorical data
Chi square
103
Risk definition
Risk has the same meaning as probability Probability is the number of times we believe it is likely to occur divided by the total number of events possible For exposure + EER= a/a+b For control - CER= c/c+d
104
Absolute risk reduction
CER-EER Absolute risk difference is the absolute change in risk that is attributable to experimental intervention Can range from -1 to +1
105
RR or Risk ratio
Ratio of risk in experimental to risk in control RR= EER/CER Assuming outcome is undesirable If RR= 1, experimental as likely as control >1 more likely in experimental <1 less likely in experimental
106
Relative risk reduction
Proportional reduction in rate of outcomes between experimental and control RRR= CER- EER / CER
107
NNT
Number needed to be treated compared with control, for one subject to experience beneficial effect NNT= 1/ARR (CER-EER)`
108
Why might absolute be valuable when given relative only
Relative can be misleading
109
Odds
The odds of an event is the ratio of number of times we believe it is likely to occur divided by the number of times it is likely NOT to occur Someone expecting a baby- Probability (risk) of it being a girl= 1/2 = 50% Odds of it being a girl= 1/1, it is as likely to be a girl, as it is not to be a girl
110
Odds ratio
Odds of the event occuring in one group divided by odds of event in another group OR= ad/bc a/b /c/d In case control: exposure is often presence or absence of risk factor, and outcome is disease presence of absence OR 1= same outcome rates OR >1 estimated likelihood of developing disease is greater in exposed than not exposed OR<1 likelihood of disease is less in exposed than unexposed
111
When do risk ratios and odds ratios differ
In general OR will always be further from the point of no effect, where OR = 1, RR= 1 If the event rate increases in treatment group, OR and RR will both be >1, OR>RR If event decreases in treatment group both OR and RR will be <1 (OR
112
Odds of cases in exposed
a/b
113
Odds of cases in non-exposed
c/d
114
What does correlation tell us
How strong the association between variables is
115
Describe scatter graph
Compare data on two variables Positive correlation= on graph points will slope from bottom lef to upper right Negative correlation= on graph from upper left to lower right Can be quantified by r= correlation coefficient
116
Correlation co-efficient
If r +ve- directly correlated, var 1 increases, var 2 increase If r = -ve, inversely correlated as var 1 increases, var 2 decreases The closer to -1 or +1, the more strong the correlation R does not correlate with the gradient of the line, rather how close the points fall in line correlation coefficients used depends on the type of data used: categorical, non-normal or normal
117
Types of correlation co-efficients
Pearson's= quantifies relationship between 2 continuous variables, normally distributed Spearman's rank= non-normal, when r is calculated using ranks. for two categorical ordinal variables, one continuous normally distributed variable and one categorical or non-normally distributed Kendall's correlation (Tau)= used for two categorical or non-normally distributed Do not establish causality
118
Regression
Used to find out how one set of data relates to another Regression line gives relationship between variables, on a scatter graph
119
Simple linear regression
Straight line that explains relationship between x and y data sets, so for a given value of x, a y value can be predicted ``` Y= outcome variable (dependent) x= independent variable a= intercept if the regression on the y axis b= regression coefficient, slope of the lie, gives strength of association ```
120
Multiple linear regression
Regression model in which the outcome variables is predicted from two or more independent variables. The independent variables may be continuous or categorical If researchers knows outcome likely to be affected by one or more confounders, not eliminated from sampling, multiple linear regression may be used
121
Logistic regression
When the outcome variable Y is binary
122
Proportional cox regression
Proportional hazards ratio, is used to assess survival or other time related event
123
Factor analysis
This is used to analyse the interrelationships between a large number of variables, and can be used to explain these variables in terms of underlying factors
124
Cluster analysis
Multivariate analysis technique that tries to organise information about variables so that relatively homogenous groups, clusters can be formed
125
ANOVA multivariate extensions
``` ANCOVA= analysis of covariance, similar to multiple regression MANOVA= multiple analyses of variable. Multiple dependent variables, multiple hypotheses testing MANCOVA= multiple dependent and independent variables ```
126
Critical appraisal in aetiological studies: case-control, cohort
1. Methodology clearly defined group? except for exposure/outcome studied, were groups similar? - selection bias, matching, restriction criteria, randomisation Did the exposure predict the outcome? - recall in cohort is issue was the follow-up complete and of sufficient duration? - attrition bias - power calculations - if too many drop outs, Type 2 error may occur were the exposures.outcomes measured in the same way in both groups? ``` 2. Results what is the RR or OR? what is the confidence limit of the estimate NNT/NNTH dose -response gradient? association make biological sense? ``` ``` 3. Applicability my patients similar to target? risk factors similar? patient's risks of adverse outcomes should exposure to risks be stopped or minimised? ```
127
Critical appraisal in diagnostic tests
1. Methodology ?was the test applied to appropriate spectrum of patients were the diagnostics test results compared to gold standard was the comparison with the gold standard test blind and independent 2. Results is the new test valid? Is it reliable? what was the outcome when patients underwent the new and gold standard test 3. Can I use this study for caring for my patients - test acceptable, available, affordable, accurate, precise in this setting - consequences of the test help your patient
128
Contingency table for diagnostic studies
``` Rows= test + / - Columns= outcome by gold standard (disease present/absent) ``` ``` a= true + b= false + c= false - d= true - ``` ``` Sensitivity Specificity PPV NPV LR +ve LR-ve Pre-test and post-test probabilities and odds ```
129
Sensitivity
Proportion of subjects with disorder who have positive result a/a+c (positive by gold standard) True positive Sensitive test when Negative rules Out disorder SnOut Sensitive test for screening
130
Specificty
Proportion of subjects without disorder who have a negative= true negative D/b+d (negative by gold standard) Sensitive test when Negative rules Out disorder Specific test when Positive rules In disorder Specific test for diagnosis
131
Generally what is pre-test probability
Prevalence Only put a person through diagnostic test if a positive result will be greater than pre-test probability
132
PPV
Proportion of subjects who have a positive result with the disease same as post - test probability of a positive result Positive test= a+b PPV = a/a+b Want PPV to be substantially higher than pre-test probability
133
NPV
proportion of subjects with negative result, who do not have the disorder NPV= d/c+d
134
LR + | for a positive result
How much more likely is a positive test to be found in a person with , as opposed to without, the condition sensitivity/1-Specificity True positive/1-true negative
135
LR - | for a negative result
How much more likely is negative test to be found in a person with, as opposed to without, the condition 1-sensitivity /specificity when < 1 means, negative test more likely to come from someone without the disease
136
Pre-test probability
a+c/a+b+c+d | Probability that a subject will have the disorder before the test
137
Pre-test odds
Odds subject with have the disorder before the test | Pre-test probability/1-pre-test
138
Post-test odds
Odds that subject has disorder after test | Pre-test odds x LR for +ve
139
Post-test probability for + results
Probability subject will have disorder after test Post-test odds/post-test odds+1
140
Does sensitivity and specificity depend on prevalence
No
141
PPV and NPV depend on prevalence/
Yes, will change as disorder becomes rarer in the population PPV will decrease NPV will increase Post-test prob will also change
142
Post test probability of negative test
not the same as NPV (probability of disorder being absent in negative test) PTP -ve= probability of disorder being present in those with negative result NPV + PTP = 100% therefore PTP -ve = 1-NPV
143
Serial testing and relationship between sensitivity and specificity
leads to increase in specificity and decrease in sensitivity | useful when treatment for disorder is hazardous and inappropriate treatment costs need to be reduced
144
Receiver operating characteristic curve
The closer the line to top left hand corner, the better the performance of the test will be= true positive high, false positive low Line of unity- a test that is no better than chance at discriminating individuals with or without disease lies on line of unity Plots true positive (sens) versus false positive (1-specificity) The larger the area under the curve, the better the test is Area of 1= perfect, 0.5 = worthless
145
Critical appraisal for treatment studies
1. Methodology clearly focused clinical question and primary hypothesis? randomisation process clearly described? concealed allocation? groups similar at the start of the study? groups treated equally apart from the experimental intervention? blinding used effectively? trial of sufficient duration? follow up complete? intention to treat study? ``` 2. Results CER EER ARR RRR RR OR NNT Precision of the estimate of treatment effect- confidence limits ``` 3. Applicability pts similar to target population? were all the relevant outcome factors considered? will the intervention help your patients? benefits worth the risks and costs? patient's values and preferences been considered? what alternatives are available?
146
Prognostic studies
looks at prognostic factors and the likelihood that different outcome events may occur Most are 1. Cohort - most prognostic - one or more followed up to see who develops the outcome - groups may classified according to the presence or absence of prognostic 2. Case-control - group with outcome are compared with a group who do not have the outcome, for the presence of prognostic factors
147
Prognostic factors vs risks
Prognostic factors are a characteristic of the patient RF increase the probability of getting a disease PF predict the course and outcome of a disease once it has developed
148
Critical appraisal for prognostic studies
1. Methodology was the sample clearly defined? sample population recruited at common point in the course of the disease? selection bias? was there adjustment for important prognostic factors> was the follow up duration sufficiently long and complete> was there blind assessment of objective outcome criteria? ``` 2. Results AR/odds RR/OR Survival analysis, survival curves Precision of prognostic estimates- confidence limits ```
149
Survival analysis, disadv
Time between entry into a study and a subsequent occurrence of an event. Technique used in longitudinal cohort studies, in which one interested in the time interval until an outcome occurs Disadv: likely not normally distributed unequal distribution periods people leave the study early, and be lost to follow up
150
Kaplan-Meier survival analysis
Looks at event rates over a study period, rather than a specific time point data presented in life tables and survival curves The survival curve will not change at the time of censoring, but only when the next event occurs
151
median survival time
time taken until 50% of the population survive
152
survival time
time fron entering into the study to developing the endpoint- time to relapse, time to death
153
survival probability
probability that an individual will not have developed an end point event over a given time duration
154
Log rank test
compare medical survival times to see any significance
155
endpoint probability
1- survival probability
156
cox regression (cox proportional hazards)
method for investigating the effect of several variables upon a time specified event takes to happen assumes the effects of the predictor variables upon survival are constant over time and are additive in one scale positive coefficient indicates a worse prognosis negative coefficient represents a better prognosis
157
Hazard
instantaneous probability of an end point event in a study degree of increased or decreased risk of a clinical outcome due to a factor, over a period of time, with various lengths of follow-up
158
Hazard ratio
comparison of hazards between two groups <1 not statistically significant= factor decreases risk of death >1 statistically significant= increased risk of death
159
4 key steps to systematic reviews
1. Specifying the question - type of study, subjects, inclusions, exclusions, intervention/exposure, outcome 2. Identifying studies - reproducible, unbiased, comprehensive 3. Extracting the data - standardised proforma, study methodology details, assessment of study quality 4. Interpreting the results - fixed or random effects, publication bias, heterogeneity
160
Why should a meta-analysis be done
Large sample size Increases power Reduces risk of Type 2 error Smaller confidence intervals
161
Weighted average
An average where the results of some studies make a greater contribution to the total than others Large weighting when: larger sample size higher event rates (estimated more precisely) Pooled result= size of combined studies
162
Forest plot
Axes - vertical= list of studies - horizontal=outcome measures, may be odds or risk ratio, means, event rates Line of no effect- for RR= 1, OR+1 Size of box= weighting
163
Variability in MA between studies can be due to
1. Chance - studies have similar and consistent results, and any differences are due to random variation - referred to as homogenous results - as a result of similarities in design/intervention/subjects, these studies merit combination 2. Systematic differences - differences between studies not due to chance - here real differences exist between the results of the reviewed studies ever after allowing for random variation - referred to as heterogenous results
164
How to determine heterogeneity
1. Forest plot= if CI of studies don't overlap, likely to be heterogeneity 2. Funnel plots, quantified using Cochran's Q, chi squared, sensitivity analysis, meta-regression
165
Chi squared statistic on forest plots
Keep null hypothesis in mind Probability of differences arising from chance= P. To calculate P, chi squared is calculated for meta-analysis Chi squared, DF, P quoted on forest plot If P<0.05, variability is not due to chance, that is, results are heterogenous. There is some methodological difference in the way that the individual studies were carried out
166
Identifying heterogeneity quickly
Use statistical tables to look up P values, compare chi squared with its degrees of freedom If statistic is bigger than DF, then there is evidence of heterogeneity
167
Z statistic
If Z > 2.2 results are heterogenous- null hypothesis can be rejected
168
Dealing with heterogeneity
If heterogeneity present, things that can be done - use a random effects model: assumes the true treatment effects in the individual studies may be different from each other. (In homogenous, used fixed effect- every study is evaluating a common treatment effect) - subgroup analysis - meta regression
169
Approaches to publication bias
``` Prevention Trial registers Trial amnesty Identification Funnel plots Galbraith plot ```
170
Shape of funnel plot
If SE on y-> larger studies at funnel on bottom, smaller sudies up top If 1/SE-> opposite is true, larger studies at the top Asymmetry of funnel plot suggests publication bias
171
r2
In statistics, the coefficient of determination, denoted R2 or r2 and pronounced "R squared", is the proportion of the variance in the dependent variable that is predictable from the independent variable(s).
172
Purpose for systematic review/meta-analysis
different studies can be formally compared to establish generalisability of findings and consistency of results. reasons for heterogeneity (inconsistencies) can be identified and a new hypothesis can be generated for different subgroups
173
Types of selection bias in SR/MA and ways to minimise
``` Publication bias Language bias Indexing Inclusion Multiple publication bias ``` Search through databases to find relevant studies`
174
How are trials weighted
1. Allocation concealment 2. Randomisation (1 and 2 minimise selection bias) 3. Blinding (measurement bias) 4. ITT analysis (attrition bias)
175
Types of hetergeneity
1. Clinical heterogeneity: introduced due to clinical differences in populations included in the study 2. Statistical heterogeneity (can detect using statistics measures) 3. Methodological heterogeneity
176
Detecting heterogeneity
1. Eye balling a forest plot- if CI all overlap, no heterG 2. Galbraith plot: ratio of log odds ratio to SE for each study, against recipricol of SE. If no stat sig heterogeneity, 95% will be within a band 2 units above and below overakk log ratio. 5% will be outside, just by chance 3.Stats: Chi squared or Q test heterogeneity df, I2 statistic calculated from cochrane Q test gives extent of heterogeneity
177
Methods to manage heterogeneity
Fixed effects model-> assumes all studies measuring same thing Random effects-> studies estimating different treatment effects Sensitvity analysis= check robustness of results, by changing parameters within the study Data transformation-> continuous to dichotomous Subgroup analyses Meta-regression analyses-can test if there's different treatment effects in different subgroups.
178
Cost benefit analysis
All effects measured in dollars Adv: easy to interpret when NB >0= new treatments extra benefits are worth more than the extra cost Dis: it is difficult to measure the value of all health outcomes in dollars ?some moral objection when not able to pay
179
Cost UTILITY analysis
Two effect= quality and length of life Product is taken as QALY Adv: outcomes involve both quality and length of life. QALY is universal, so easily compared Dis: QALY measured vary by method May vary by respondent Society may value a QALY for different patient group differently
180
Cost effectiveness analysis
Adv: One effect measured in "natural units" incremental cost effectiveness Dis: Only one outcome will represent the effect of treatment, however other outcomes may be relevant
181
Cost-minimisation analysis
Not worried about outcomes Adv: only need to collect cost data Dis: Few treatments have identical outcomes Researchers likely need to collect the effect data to verify the equal effect assumption
182
Advantages and limitations of economic analysis
Adv: Systematic evaluation of costs and consequences Answers the question: Is the intervention worth the cost, including if it is cheaper than the other comparator, decision makers can render judgments Better advocacy in health care Makes decision making explicit Can be used to guide priority setting Limitations: The primary limitation of summarising cost and consquences in an ICER 'price tag' is that decision makers, assuming they find the economic evaluation useful, may still decide whether the extra gain associated is worth the extra cost. Does not explicitly consider decision makers budget May over simplify health decisions
183
CEA is only done when, and calculation
When the intervention is either more expensive and more effective, or less expensive and less expensive ICER= C1-C1/E1-E2 All should have sensitivity analysis done, to test the extent to which changes in the parameters used in the analysis may affect the results obtained
184
An intervention results in a patient living for an addition 4 years , rather than dying within one year, but where QOL reduced from 1 to 0.6 will generate
1. 4 x 0.6= 2.4 2. less 1 year @ reduced quality= 1-0.6= 0.4 3. QALY's generated = 2
185
Advantages and disadvantages of QALY
Adv: combine estimate of extra quantity and quality of life provided by the intervention in one measure Can compare interventioins or programs in same therapy area Making healthcare decisions and allocation of resources Setting priorities with respect to healthcare interventions Disadvantages: Values assigned may not reflect values of patients receiving treatments May lack sensitivity within disease area May be derived from population studies that may not be generalisable to the population you are treating 20-40,000 fir 1 QALY or one DALY averted, usually considered cost effects
186
How to calculate incremental net benefit
Cost effectiveness analysis INB: (extra effect x willingness to pay) - extra cost
187
Interpreting CEAC
represents uncertainty in cost effectiveness analysis Indicates the probability that the intervention is cost effective as compared with the alternative.
188
Bootstrapping
Commonest method used to construct CEACs | Constructing CI and the visually representing it as a CEAC
189
Methods in qualitative research
1. Grounded theory- used to develop a theory, 'grounded' in the groups observable experiences 2. Phenomenological approach- to gain a better understanding of everyday experiences of a group of people 3. Ethnographic approach- learns about culture by observing people from that culture
190
Data sampling methods in qualitative research
1. Purposive: purposefully selecting wide range of informants to explore meanings and also select key informants with important sources of knowledge 2. Theoretical sampling- type of purposive, developing a theory or explantion guides the process of sampling and data collection. Analyst makes initial sample, codes, collects and analyses data, and produces a preiliminary theory, before deciding which further data to collect 3. Convenience sampling 4. Snowball- target populations are elusive, participants asked to identify others with direct knolwedge relevant to the study 5. Extreme case sampling- participants chosed because of their knowledge or experience is atypical or unusual in come way relevant to the study being conducted
191
Data collection in qualitative, triangulation
1. Interviews 2. Focus groups 3. Participant observation Triangulation-> multiple data gathering techniques or multiple sources 1. Investigation data method
192
Data saturation
despite further data gathering and analysis, understanding is not developed further. At this point, data collection and sampling ends
193
Data analysis in qualitative
1. Meaning focused: code relevant themes within data. Understand experiences and meaning 2. Discovery focussed: analysis of segments of text, coded, sorted and organised, looking for patterns or connections 3. Constant comparison: test and re-test Clear transparent process= audit trait
194
Minimising bias in qualitative
1. Transparence 2. Bracketing (exclusion of preconceptions) 3. Reflexivity: researchers aware of own preconceptions. Reader can weigh researcher's role in the conduct of the study 4. Member checking: researcher returns to one of more participants to check the researcher's interpretations of what the participants have said