Flashcards in Final Exam Deck Deck (31)
How do objective versus subjective endpoints (outcomes) influence the reliability and validity of a RCT?
The RCT becomes more reliable and valid as the endpoints move from subjective to objective.
How does one distinguish whether the designated outcomes of a RCT are objective or subjective?
Objective outcomes - Are defined in explicit terms that leave no ambiguity. There is limited personnel for assessing outcomes, and explicit procedures for dealing with conflict.
Subjective outcomes - Surveys and scales are used. Validity is established by assessing whether the outcomes are met, and reliability is established by whether the assessors have been properly trained and are limited in number. MCID is established.
Why is it important to establish a minimal clinically important difference (MCID)?
MCID is the minimal amount of difference that is important to patients. Statistically the difference may be meaningless, but to patients the difference is important. There are limitations, however. The consensus of the difference is not determined by patients, the assessment used to determine the MCID may be subject to recall bias, and the change is statistically determined.
Would we typically identify the MCID in studies with objective or subjective endpoints?
Subjective endpoints determined through scales and surveys.
Is it possible for the results of a study demonstrate statistical significance but not be clinically important? Both statistically AND clinically significant? Neither? Show how by using a diagram which includes the MCID, point estimate & CI. Be sure to point out the direction of benefit and harm with respect to the line of no difference.
Statistical significance w/o clinical significance: When the confidence interval does not cross the null value, but the MCID is greater than the CI.
Statistical significance w/ clinical significance: When the confidence interval does not cross the null value, and the MCID value is lower than the CI (on the left, or w/i the CI on the leftish side).
Neither statistical significance nor clinical significance: The CI crosses the null value, and the MCID value is outside of the CI to the right.
What are the advantages and disadvantages of using surrogate endpoints in a RCT?
A surrogate endpoint is used in place of a clinically meaningful endpoint.
Advantages: Fewer patients, shorter trial/faster to market, easier to measure, less invasive for patients.
Disadvantages: Trial is too short to evaluate longterm effects, intervention may effect surrogate endpoint but not clinical outcome, and patient may have a hard time relating the surrogate to the clinical outcome.
What are key questions must be addressed when evaluating composite outcomes in a RCT?
Are the components of the outcomes related physiologically so that intervention affects them similarly?
Are the components related such that they are of equal importance to patients?
Is the incidence of the components similar?
Is the treatment effect similar for all components?
Compare and contrast P-values and confidence intervals in terms of their helpfulness in establishing RCT reliability.
P-values indicate precision in the abstract, but does not provide good variability or information about clinical significance. Less desirable than CI's.
CI's also provide information about precision in rejecting the null, and about variability. There is some information about clinical significance, and CI's are more desirable.
Give an example of how results from a RCT might be statistically non-significant but trend toward a clinically meaningful benefit. What type of error might this scenario represent?
What about statistically significant but not clinically significant?
1. This might indicate a Type II error. The results may show that the intervention will not provide a statistically significant outcome for the patient, but the patient may feel that the intervention works and so the clinical effect of having the intervention is meaningful to the patient. This will have a narrow confidence interval.
2. This might indicate a Type I error. The results from a RCT may be statistically significant in that the confidence interval does not cross the null, but the outcome may not be clinically meaningful for patients. For example, a new antibiotic comes out that statistically will help the patient, but the difference is so minimal that the patient does not feel better.
In a non-inferiority trial such as “ESTABLISH-1” what is the purpose of delta? (Δ)
Delta is the clinically acceptable difference. In the ESTABLISH-1 trial (a non-inferiority trial), having a confidence interval w/i the delta means that the treatment is non-inferior or equivalent.
For a non-inferiority trial, designate the CI, delta, and ARD on a graph, and interpret the finding.
CI - The range that shows outcome significance.
Delta - The value that is the clinically acceptable difference
ARD - (Absolute risk difference) The range from -delta to +delta that allows the treatment to be non-inferior, equivalent, inferior, or superior.
For a non-equivalency trial, the treatment is non-inferior as long as it does not cross the (-) delta. It may cross the (+) delta to be superior, and that still counts as non-inferior. This is different from an equivalency trial, which requires that the CI be w/i both the (-) and (+) delta.
How is the null hypothesis in an Equivalence/Non-Inferiority RCT different than in a Superiority RCT design?
In a superiority trial, the null hypothesis means that there is no difference between treatments. In an equivalent/non-inferioirity trial, the null hypothesis means that there is a difference between treatments.
What are some of the issues with multiplicity in a RCT?
Each time there is a new comparison, it increases the chance of a Type I error (where you think there is a difference between groups when there really isn't any)
What are some important considerations when evaluating primary and secondary outcomes in RCTs?
Primary outcomes: Should be clinically relevant. Endpoints should be identified before the analysis.
Secondary outcomes: Should be related and support primary outcomes. Usually involve safety. Explore rather than conclude.
When is statistical adjustment needed for multiple outcomes or multiple arms within a RCT?
It is rarely needed for well-designed studies.
It is used when there is more than 1 primary outcome a priori.
It is used when there is a post-hoc change of primary outcome.
It is used when secondary outcomes are being seen as conclusive.
What are some considerations for the appropriate planning, analysis, and reporting of subgroup data to preserve the reliability of a RCT?
- Should be pre-specified (identified a priori)
- Clinically justified
- Confined to primary outcome and limited number of subgroups
- Should report all analyses done, not just significant ones.
- Should use statistical tests of interaction to test for effect modification rather than individual p-values
- Rarely should subgroup analysis effect the trial's conclusions.
Define interim analysis and what should reviewers look for if it is present in a RCT?
An interim analysis is an analysis done in the midst of a trial by an independent committee at pre-specified check points and stop the trial for safety or futility.
Be cautious of these!
Multiple interim analyses contribute to multiplicity.
Look out for:
Inadequate power size
Stopping early indicated by incomplete trial size.
A statement that no interim analyses were done (rare)
Trial ended early without pre-specification
Define the various types of bias: selection, performance, attrition, detection, reporting
Selection bias- When the selection of the treatment groups was not not done equally.
Performance bias - When the treatment groups are treated differently during the course of their treatment.
Attrition bias - When the patients drop out due to different reasons. This results in the analysis groups being unequal.
Detection bias - Difference in how the outcomes were assessed. This can happen if the outcomes aren't determined a prior or if those analyzing are not blinded.
Reporting bias - When the participants knowingly falsify data based on a stigma.
Explain some benefits and limitations of using RCT assessment tools in terms of the 4 study evaluation domains.
There are four study domains: Validity, Reliability, Applicability (or external validity), and Clinical Importance. RCT assessment tools are focused on internal validity only.
Does randomization minimize or eliminate selection bias? Why?
It minimizes bias when the potential confounders are distributed equally between treatment groups. There is a need for allocation concealment and valid randomization procedures. There can still be some bias if the randomization is imbalanced, and this can be minimized again with a large n value.
What trial size is needed to limit chance imbalances?
At least an n of 200. (Both simple and restricted randomization)
Give some examples of non-random allocation.
If the randomization occurs by a characteristic that a patient has (such as birthdate, age, appearance, date of test result) the groups are less likely to be random.
Explain some good allocation concealment methods.
- Use central allocation (IVR, web-based, pharmacy-controlled randomization)
- Use sequentially numbered, opaque, sealed envelopes
- Use sequentially numbered drugs that are identical in appearance
Compare and contrast allocation concealment and blinding.
Allocation concealment: Occurs before trial, always achievable, leads to selection bias if not performed
Blinding: Occurs during trial, not always achievable, leads to performance and/or detection bias if not performed.
Is blinding always possible? Explain.
No. There are unique side effects profiles of some drugs that are difficult to hide, and some surgeries are also difficult to side. Some ways around this could be sham surgeries, where only the patients are blinded (one-sided blindness)
What are some issues with unblinded assessment of outcomes and how can these problems be overcome?
Unblinded assessments of outcomes are prone to performance bias, where the physician treats the patient according to their outcome.
These can be overcome by :
- Single blind
- A priori criteria with established set training
Why does Intent-to-treat help control for attrition bias?
It preserves randomization, and assumes that people drop out for a reason and not randomly. It represents treatment effect in real world conditions. Assumes that the patients did not have the effect in superiority trials. (Most conservative estimate of treatment effect in superiority trials)
Discuss the differences between Intent-to-treat, Per Protocol, “Modified” ITT, and “As treated” analysis.
ITT- Want patients to be analyzed in original groups, and for all patients to be analyzed. Inclusive.
Per protocol - Includes only patients that had optimal adherence to protocol. Not real-life. Loses randomization. Appropriate for safety analyses and non-inferiority trials.
MITT- Excludes patients for reasons not justified.
As treated - Loses randomization because they analyze patients according to group.
Demonstrate how to calculate and interpret overall and differential attrition rates for a RCT.
Overall attrition: Add up all the people (all treatment groups) who dropped out of trial. Divide by total number of people that were initially randomized. Rate should be less than 20% to have little attrition bias. (Or greater than 10% between groups)
Differential attrition: The attrition rates of both treatment groups should be calculated and then compared. They should be similar, and the reasons for leaving the trials should be similar.