Brainscape's Knowledge GenomeTM


Top Flashcards (Certified)
All Flashcards

BCOP > Stats, Trial Design, Interpretation > Flashcards

Stats, Trial Design, Interpretation Flashcards

1
Q

Internal validity

A

 How is the study structured?

 Is it a “good” study?

 Study design issues

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

External Validity

A

 Does the study apply to my situation?

 Is it applicable to the patients I see?

 Is it practical?

 Generalizability

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Types of Study Design -5

A

 Descriptive

 Observational
 Case control
 Follow-up
 Cross-sectional

 Experimental

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Types of Study Design - descriptive -3

A

No comparative group– no intervention

ex. case study, case series, survey, education intervention with no comparator

can be large (ie. 30,000 high dose theophylline)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Types of Study Design - observational (epidemiological) (think watchful scientist)

A

Comparative group; no intervention

  1. case control: based on outcome
  2. follow-up: based on risk factors
  3. cross-sectional: hybrid of the two
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Types of Study Design - experimental -7

A

Comparative group

 Patients selected

 Consent

 Investigator allocates

 Intervention

 Measurements

 Assess outcome

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Types of Study Design - experimental - Parallel vs Crossover - PARALLEL DEF-4

A

 Each patient receives one therapy

 Two concurrent groups

 Interpatient variability

NEED MORE PTS

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Types of Study Design - experimental - Parallel vs Crossover - CROSSOVER DEF-5

A

 Each patient receives one therapy then another

 Randomized to sequence (everyone gets both drugs)

 Tx A -> outcome -> Washout (5 half-lifes) ->Tx B -> outcome

 Position effect (statistics)

FEWER PATIENTS NEEDED

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Types of Study Design - experimental - ADVANTAGES -4

A

 Control more variables, can blind

 Decrease sources of bias

 Ascertain cause and effect (can not say that in other trials)

 “Cadillac” of study designs

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Analyzing Methods Section - FLUFF

A

 Types of bias

 Often use flowcharting to follow a patient through the study

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Selection Bias -5

use Table 1

A

 Was bias introduced in how the patients were selected?

 Is the study population adequately defined?

 Inclusion and exclusion criteria

 Treatment groups comparable

 See “Table 1” of study

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Classification Bias def -2

A

Refers to how classifications made (bias can be made in recruiting pts, often defs in supplementary material, definitions can be extensive)

 ex. Postmenopausal, Receptor positive, Outcomes—disease-free—survival event

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Preventing Classification Bias -3

A

 Use structured definitions

 Use “reliable,” “complete” sources of information

-is EHR good source of info

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Allocation Bias def -2

use Table 1

A

 Was bias introduced when patients assigned to their groups? - very hard to assess because studies often just say “pts were randomized”

 Was it truly random? use Table 1 to see if equal

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Randomization method -1

A

 Permutated blocks (for every 4 pts, assign 2 to a group -> this allows study to stop in middle if needed)
 Keeping even numbers of patients in the groups throughout the conduct of the study (to allow better stats)
 Stratified according to participating center
 Chemotherapy planned to be given before, during or not at all

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Compliance Bias def -3

A

 How was compliance assessed?

 Not ALWAYS specifically addressed in study - this makes it HARD to assess

 ie. Semiannual visits for first 5 years

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Attrition Bias def -3

A

 Drop-outs and why (acct for all pts)

  • ie. may just drop out (withdraw consent) or be ineligible following medical review

 If more patients drop out of one group vs another, does this introduce bias or influence the results?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Interventions def -3

A

 Comparable

 Blinding
Double-blind: Neither investigator nor patient knows patient allocation
 Single-blind: Either patient or investigator does not know

 Competing interventions (that would influence results)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Observer and Measurement bias -5

prevent with blinding

A

 How are outcomes measured?

 Is it appropriate?

 Patient or observer influences

 Sufficient observation (challenging, need many yrs for oncology)

 Is it clinically meaningful?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

Confounding Bias -4

all studies susceptible

A

 Attributing the outcome to a risk factor not related to the outcome (wrongly attributing outcome)

 Can control for many variables in the analysis

 Often difficult to prevent

 Look at exclusion criteria

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Preventing Other Problems - Is the study powered to be meaningful? -3

A

 Study enough patients

 Discuss with statistical power

 Usually discussed when sample size calculations presented in methods

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

Analyzing Results

A

 Add numbers to flow chart (assess attrition)

 Follow the numbers

 Attrition

 Present results for EVERYTHING mentioned in methods

 Statistics

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

What Data To Include - Intention to treat

A

All patients randomized included in analysis

Considered the most conservative analysis

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

What Data To Include - Modified intention to treat

A

all patients randomized AND received at least one dose of therapy

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
What Data To Include - Per protocol
Only those patients who completed the study per protocol (ie, pt's dropped if ADR and stopped tx) For many studies useful to have both an intention to treat method and a per protocol
26
Statistical Analysis -Descriptive -3
 measure of central tendency mean, median, mode, etc.  spread of the data
27
Statistical Analysis -Inferential 1
Null hypothesis =No difference exists
28
Non-inferiority Trials -3 &&& new in last 10 years hard
 Use a different hypothesis: The two treatments are not non-inferior to each other (tough because double negatives) (think the treatments are same)  P values mean different things—if less than 0.05, means they are non-inferior  Can’t claim superiority with these trials but can do a non-inferiority analysis then a superiority analysis
29
Non-inferiority Trials -Why? -4
 Unethical to do a placebo controlled trial  Treatment expected to be similar to standard treatment (Therapeutic non-inferiority to active control)  Treatment assumed to be better than placebo  Treatment likely to have other advantages (safety, cost, convenience . . .)
30
Non-inferiority Trials -set up -3 no clue here
 Set a “marginal difference”  Uses alternative hypothesis  Set confidence interval threshold -actually need more pts for this type of trial
31
study types - superiority vs equivalence vs non-inferior def -3
 Superiority trials (Is new therapy significantly better or worse?)  Equivalence trials “neither any better or any worse” (Establish equivalence range. Is it in the range to be similar?) (to see if 2 drugs are pharmaceutically equivalent)  Non-inferiority “not much worse than the active comparator” (Is new therapy no worse than control?)
32
Statistical Tests - Nominal: yes or no - def and examples
 Categorical  Response rate (patients responded or not)  Adverse events or not  Alive or dead  Pregnant or not  Race
33
Statistical Tests - Nominal Data “Traps” -2
 Percentages  Seem like on a continuous scale  Think of the data origin  Did the patient have a response or not?  Response is yes-no  Presented as % patients with response  Multiple groups or categories  Still assess if yes or no they belong to each group  No ranking
34
Statistical Tests - Nominal Data Tests -4
 Chi-Square (lots of rules)  N>40  20-40 use if expected frequency of cells >5  Fishers exact (if N<30 use but can use for all nominal data)  Related samples: McNemar (cross-over)  3 or more independent groups: Chi-Square (also called Chi-Squared for independent groups)
35
Statistical Tests - Ordinal Data def -6
 Ranked  Likert scales (strongly agree to strongly disagree)  Hierarchy  Responses not mathematically equal  ie. Years of HRT (none, 0-5 years, 5-10 years, greater than 10 years), Age of diagnosis (less than 50, 50-55, 55-65, older)
36
Statistical Tests - Ordinal Data “Traps” -4
 Likert scale (1-5, strongly disagree-strongly agree)  Calculate means  Behaves like continuous data and presented as continuous—need to remember still ordinal data!!  Useful to present median, mode, “top box” = positive responses like Likert 4 and 5 only -USE MEDIAN NOT MEAN
37
Statistical Tests - Ordinal Data Tests -4
 Mann Whitney U test (based on MEDIAN)  Wilcoxon rank sum test  Related samples (cross-over)  Sign test  Wilcoxon signed rank test  Kruskal-Wallis ANOVA (for multiple groups)
38
Statistical Tests - Continuous Data -Continuous “Traps” -2
 Data presented as % probably not continuous  Are composite scales, etc, really continuous?— many times yes!  Battery of ordinal scales  Total (when all batteries combined together) behaves as continuous
39
Statistical Tests - Continuous Data -def and examples
 Interval, ratio data  Time to disease progression  WBC, platelet count  Serum creatinine  age, weight
40
Statistical Tests - Continuous Data: VAS -2
```  Visual analog scales (VAS)  Scale of 0-10, 0 being no pain, 10 being the worst pain imaginable (DO NOT DEFINE THE POINTS IN BETWEEN)  Only anchors the ends  Administered verbally or in writing  Handled as continuous data ```  Other pain scales: 0=no pain, 1=mild, 2=moderate, 3=severe  Defines all points  Handled as ordinal data
41
Statistical Tests - Continuous Data Tests -5
 Parametric vs Non-parametric  Mann-Whitney U (median)  Student’s t-test (2 groups) (mean)  Normal distribution (are both mean and median similar), equal variance (are std deviations the same)  Related Data: paired t-test (cross-over)  ANOVA (3 or more groups)
42
Hypothesis Testing -4
 Start with null hypothesis  Superiority trial: There is no difference  Equivalence: The groups are not equivalent  Non-inferority: The therapy is not non-inferior to the other therapy
43
Types of Error—Superiority columns across are truths rows are experiment
1. Type I or alpha error (alpha, p value) TOP RIGHT OF BOX = experiment shows "difference exists" WHEN IN FACT truth is "no difference". alpha is set up front as 0.05, P VALUE DETERMINED AFTER STUDY AND IS NEW ALPHA AND TELLS IF SIG 2. Type II or beta error (beta) BOTTOM LEFT OF BOX= experiment shows "no difference exists" WHEN IN FACT truth is "difference". beta is set up front. 0.2 is good, some do lower UNFORTUNATELY NO EQUIVALENT P VALUE TO FIGURE WHERE WE REALLY FELL
44
Types of Error—Superiority columns across are truths rows are experiment
1. Type I or alpha error (alpha, p value) TOP RIGHT OF BOX = experiment shows "difference exists" WHEN IN FACT truth is "no difference". alpha is set up front as 0.05, P VALUE DETERMINED AFTER STUDY AND IS NEW ALPHA AND TELLS IF SIG 2. Type II or beta error (beta) BOTTOM LEFT OF BOX= experiment shows "no difference exists" WHEN IN FACT truth is "difference". beta is set up front when doing sample size. 0.2 is good, some do lower UNFORTUNATELY NO EQUIVALENT P VALUE TO FIGURE WHERE WE REALLY FELL
45
Power and Sample Size -5 picked at the begining
 Power = 1 - beta  Determined by alpha (p value) and beta values desired  Estimated response rate  Difference believed to be valuable  Front-end concept!!
46
Sample Size Calculations using power
slide 80 &&&
47
Expressing Risk - three types and data used -3
 Expressed as odds ratio, relative risk or hazard ratio  Used for nominal data ONLY!!!  Use a 2x2 table—helpful for organizing data in study
48
Odds Ratio def and trial use -3 ESTIMATE OF RISK
Based on prevalence No denominator, making assumptions Case Control, cross-sectional
49
Relative Risk def and trial use -4
Based on incidence Denominator Association between exposure and disease over time Follow-up, experimental
50
Incidence -2 PREVALENCE IS WITHOUT UNIT OF TIME
 (Number of persons developing dx/ total at risk) per unit of time  Direct estimate of probability or risk
51
Relative Risk rationale and calculation -5
 Expression of risk for follow-up studies (also experimental trials)  Accounts for denominator information  Calculation: RR = (a/a+b) / (c/c+d)  The proportion between the two!!  Usually presented with a confidence interval
52
Interpreting Risk (by outcome) -4
 1 = no difference between the groups  2-5 = mild association  5-10 = moderate association  > 10 = strong association
53
Relative Risk Reduction -3 RELATIVE BENEFIT INCREASE
 The most “optimistic” way to present risk ie.  Calculated 1-RR = 1-0.82= 0.18= 18%  Letrozole decreased the risk of a disease free survival event by 18%
54
Absolute Risk Reduction -3 ABSOLUTE BENEFIT INCREASE
 Takes into account the actual values of the numbers rather than just the proportion  Are we talking events that occur 1 in 10 or 1 in 1000?!! ie.  Calculated (A/A+B)- (C/C+D) =8.8%-10.7% = 1.9% (absolute value) -NOTE INCIDENCE DIFF
55
Number Needed to Treat (NNT) calculation and use-3 VERY IMPT -KNOW
 Inverse of ARR  Be sure to convert percentages to decimals  Way to make numbers more practical and meaningful  Concept is “Number Needed to Harm” (NNH) for adverse events -ALWAYS ROUND TO NEAREST WHOLE PERSON
56
Survival Analysis -5 &&& BASICALLY RELATIVE RISK AMPED UP EX. BREAST CANCER AND HORMONE EVENTS JUMPED AFTER 5 YRS - THINGS CHANGE OVER TIME
 Takes into account the timing of events  Weighted relative risk over the entire study  Result is Hazard Ratio (HR)  Data presented in Kaplan—Meier curves  Cox proportional hazards regression the most common for multivariate analyses; log rank test for differences in survival
57
example need for survival analysis - 2
 Takes into account that you had many patients for the first two years, and not as many in the last 3 years  Same number of events in end, but different denominators over time
58
Censoring def -1 and examples -4 -slide 101&&&when is arm favored??
 Accounting for missing or incomplete data  Study ends before patient has an event  Patient is lost to follow-up  Patient withdraws due to an adverse event  Patients voluntarily crossed over to other tx (ie. letrazole)
59
IPCW: Inverse Probability of Censoring Weighted analysis DEF -1
Modeling technique to account for bias introduced in censoring
60
Censoring vs ITT issues -3
 ITT: Tamoxifen looks better than it may be (patients that crossed over to letrozole included =pt likely had better outcomes)  Censored: Only disease free patients allowed to cross-over. High risk patients left in tamoxifen group.=pts likely had worse outcomes -USE IPCW analysis to account for adjust for these biases
61
RR vs HR &&&slide 103 clarify
 Relative risk can easily be calculated from numbers presented in the study  Hazard ratio is the same concept but is the weighted relative risk over time  Adjusts for change over time  Adjusts for “repeated measures”  Adjusts for different “slopes” of the line
62
P-Value def -3
 Probability results due to chance alone  Determine level of significance (alpha value) prior to conducting the study  By custom, p < 0.05 is considered “statistically significant”
63
Statistical Pearls as related to p value -4
 The size of the p value has nothing to do with the importance of the result (ONLY YOU DETERMINE THIS, ie. p=0.001 for bp med which measured 2mmHG diff)  Do not confuse statistical significance with clinical significance  Results that are not statistically significant MAY still be important Statistics do not determine what is important, statistics determine how certain we are.
64
Confidence Interval -5
 95% CI (If study was repeated 100 times, 95% of the time the result would likely fall in this range)  Provides a “range” to result (Inferences on the population) (DO NOT CONFUSE CI WITH STD DEV, which only tell you about variation in study)  Calculation based on Standard Error of Mean (SEM  CI can be applied to any type data  IMPT TO Determine value that represents no difference  When used with OR, RR or HR  No difference value = 1 (If CI doesn’t include 1, then statistically significant)
65
Other Statistical Issues: Repeated Measures - Cox model - 3
 If made multiple measurements over time, then need to correct for it using a statistical test that takes into account repeated measures  ie. Evaluated every 6 months  Cox model accounts for this
66
Other Statistical Issues: Bonferroni Effect -3 &&&clarify slide 113 NOT CLEAR
 If look at enough things, something will be statistically significant just by chance alone  If didn’t make correction and should have, multiply the p value by number of comparisons.  2009 states adverse drug reaction (ADR) analysis not adjusted for multiple comparisons
67
WHEN IS IT POSSIBLE TO MAKE A BETA ERROR?
When p value is >0.5 because you are saying there is NO DIFFERENCE BETWEEN THE GROUPS.
68
Duration of Studies
 Were patients studied for sufficient duration?  Do the results change over time?  Are the same things being compared at each time point? &&&
69
Subgroup Analysis key points -4
 Allocation no longer applies (NOT RANDOM)  Sample size calculations don’t hold for subgroups (POWER NOT APPLIED)  As more subgroups evaluated, more opportunity for finding a significant result when one does not exist  Results can be overstated and misleading
70
``` Interpreting Forest Plots when used? -2 what does bar mean? -1 what does box mean? -2 do shorter bars usually have larger boxes? ```
 Used with subgroup analysis and meta-analyses  Bar = confidence interval  Box  Location = HR  Size = number of people in analyses  Usually shorter bars have larger boxes = smaller confidence interval as increase sample = more confident of result
71
Meta-Analysis
 Combine results from many studies  Reanalyze  Decrease beta  Specific criteria for selection and classification of studies (selection bias refers to how studies selected!!)  Studies should have similar methodologies  Compounds problems observed in the individual studies
72
Effect Size slide128-130&&&never heard of this NOT COVERED IN VIDEO
 Way to standardize the effect  Used for continuous data with normal distribution  Calculated by dividing the difference of means by standard deviation  Use table from website to interpret and make more practical
73
Reporting Data key concepts &&&clarify
 How reported affects significance placed on data  Watch graphs!! (CAN BE MISLEADING)  Changing numbers to %  Collapsing data in categories  % change from baseline
74
Case-control Studies def -3
 Identify cases with the disease of interest (outcome)  Identify controls without the outcome  Look back in time (from present to past) to assess the risk factors
75
Retrospective study. . . .key concepts -3
 Often confusing terminology  Study design: another name for case control  Refer to the time frame of the study
76
Application of Case- Control -4 what to apply to? "rare" yes or no? expensive?
 Applied to new diseases or outbreaks  Can study “rare” diseases  Evaluate multiple risk factors  Relatively easy and less expensive
77
Weakness of Case - Control, aka types of bias -4
 Selection bias  Classification bias  Information bias&&&  Confounding bias&&&
78
Cross - Sectional study def -4
 Identify a study population (from present going forward)  First Classify based on outcome  Second separate outcomes to Classify based on risk factor  Predict prevalence
79
Cross - Sectional Problems / weaknesses -4
 Chicken and the egg  Confounding bias &&&  Selection bias  Classification bias
80
Follow-up Studies def -5
 Identify a study population (from present to future)  Exclude individuals with the outcome of interest  THEN those without outcome of interest -> Classify based on risk factor  Follow over time  Assess outcome
81
Follow-up study features -4 | why best?
 Strongest study design  Strongest causal link  Denominator; predict incidence  Can usually address information bias &&&
82
Follow-up bias / weakness -4
 Hawthorne effect &&&  Surveillance bias  Change over time  Attrition bias &&&
83
Issues in Oncology Studies -6
 Duration of therapy and evaluation  Results represented  Endpoints selected  Combination therapy  Doses, regimens, routes  Balancing cost and clinical outcomes
84
basics of statistical tests -3
type of data (nominal, ordinal, continuous) number of groups independent (parallel) OR related (cross-over) groups

BCOP (32 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2025 Bold Learning Solutions. Terms and Conditions