Study Design Flashcards

1
Q

What are the Types of Research?

A

Quantitative Research, Qualitative Research and Mixed-Mode

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Types of Research: What is Quantitative Research?

A

Generates numerical data, or data that can be converted to numbers

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Types of Research: What is Qualitative Research?

A

Explores and understands people’s beliefs, experiences, attitudes, behaviours and interactions. It generates descriptive, non-numerical data

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Types of Research: What is Mixed Mode?

A

A combination of quantitative and qualitative research

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What are the 2 Types of Quantitative Studies?

A

Observational and Experimental/Interventional

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Quantitative Studies: What is Observational Study?

A

Where the researchers do not intervene with any treatment (only record data and report on patients history etc)

We’re just observing what happened and reporting the outcomes

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Quantitative Studies: What is Experimental/Interventional Study?

A

Where the effects of new treatments are examined

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Quantitative Studies: Observational Study - What are the 4 Types of Observational Studies?

A

Case Reports/Series

Cross-sectional Studies

Case Control Studies

Cohort Studies

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Quantitative Studies: Experimental/Interventional Studies - What are the Types of Experimental/Interventional Studies?

A

Randomised Controlled Trials (RCTs)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Evidence Can be Classified as What?

A

Evidence can either be classed as Level 1, 2 or 3, with 1 being the best

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Evidence: What is Level 1 Evidence?

A

Randomised Controlled Double Blind Studies, Systematic Reviews and Meta-analyses

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Evidence: What is Level 2 Evidence?

A

Case Series, Case Control Studies, Cohort Studies

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Evidence: What is Level 3 Evidence?

A

Case Reports, Ideas/Editorials/Opinions

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Observational Studies: What is a Case Report/Series?

A

Report on a single patient (report) or collection of patients (series)

It is a descriptive study launched when an unusual, rare or novel occurrence happens with a drug and we want to find out why this event occurred.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Observational Studies: What are Advantages of a Case Report/Series?

A

Identify new and unusual events

Generate hypotheses (i.e. lead to other, bigger studies)

Simple and quick to publish

Basic, quantifiable data

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Observational Studies: What are Disadvantages of a Case Report/Series?

A

No control group (we don’t know what else is going on with the patients, like risk factors etc.)

Lack of statistical power to draw conclusions (harder to get numerical conclusions)

May be misleading

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Observational Studies: What are Cross-Sectional Studies?

A

In these studies, we are taking a cross-section of the population to give the prevalence of a particular outcome at that point in time. These studies are a snapshot of the frequency and characteristics of a population at any given time and allow us to estimate the prevalence of the outcome of interest for that population

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Observational Studies: What are Advantages of Cross-Sectional Studies?

A

Quick, easy, cheap

Very good at estimating prevalence of outcomes, since sample is taken from the whole population

Many outcomes and risk factors can be assessed

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Observational Studies: What are Disadvantages of Cross-Sectional Studies?

A

Difficult to make causal inference (can’t see ‘why’)

May be prone to non-response bias (some don’t respond)

Only a snapshot, hence only true on that day

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

Observational Studies: What are Case Control Studies?

A

These studies are done ‘backwards’. They begin with people with the outcome, and people without the outcome. Participants are then matched for age and sex, then we go back and see what made them different to find why the outcome occurred. Information is collected on a range of exposures, via a questionnaire that everyone has to complete asking about risk factors and potential confounders etc

Comparing histories of participants with and without the outcome, then drawing conclusions based on any differences found between the two groups

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Observational Studies: What are Advantages of Case Control Studies?

A

Good for rare, slow developing conditions (since we are looking backwards)

Less time needed to conduct the study – don’t have to wait for people to get disease

Can look at multiple exposures and risk factors at once. If we were to start from the start, we would not know what to look for.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

Observational Studies: What are Disadvantages of Case Control Studies?

A

Always retrospective, causing challenges in valid data collection (poor memory of some people, especially since event may have happened years ago)

Difficulty finding suitable controls

Potential recall bias (related to memory issue since events happened a while ago) as well as selection bias.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

Observational Studies: What are Cohort Studies?

A

Cohort studies consist of a large group of exposed people followed over time, compared to a group that has not been exposed. Information is collected on a range of events to find the outcomes of being exposed to the exposure(s). It is essentially opposite to case control, since it is prospective in nature

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

Observational Studies: What are Advantages of Cohort Studies?

A

Can detect a range of outcomes

Minimise selection bias, since we start with many people and none have the outcome at the start

Removes risk of recall error

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

Observational Studies: What are Disadvantages of Cohort Studies?

A

Requires large populations as the incidence is unknown – how rare is it?

Can take a long time and are expensive

Susceptible to loss to follow-up or withdrawals

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

Interventional Studies: Clinical trials are used in drug development to achieve the best balance between the three following objectives:

A

Gaining the best possible evidence of efficacy and safety

Ensuring the safety of participants

Cost and time minimisation for drug development

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
27
Q

Interventional Studies: Why are Clinical Trials are used in Drug Development?

A

We do these trials to identify safety issues not apparent in pre-clinical testing (i.e. in animal testing, or in vitro) and also to provide evidence of efficacy before the drug is released to the market

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
28
Q

Interventional Studies: Clinical trials in drug development can be divided into 5 phases, What are these Phases?

A

Pre-clinical

Phase I

Phase II

Phase III

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
29
Q

Interventional Studies: Clinical trials in drug development can be divided into 5 phases. What is the Pre-clinical Phase?

A

In vitro testing, animal testing. Identifies safety issues and some evidence for starting doses

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
30
Q

Interventional Studies: Clinical trials in drug development can be divided into 5 phases. What is the Phase I Phase?

A

Usually conducted in healthy volunteers (up to 200), sometimes with a small placebo group, and start with low doses before increasing. Is aimed at finding out safety, PK, and determining safe doses for the next phases

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
31
Q

Interventional Studies: Clinical trials in drug development can be divided into 5 phases. What is the Phase II Phase?

A

Drug is introduced into population of interest (i.e. people who have the disease), usually with more participants and a placebo group. The primary interest is safety, while the secondary consideration is efficacy. We can also get an idea of different doses through this phase

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
32
Q

Interventional Studies: Clinical trials in drug development can be divided into 5 phases. What is the Phase III Phase?

A

Randomised Controlled Trials (RCTs). Placebo controlled trials, or active comparator trials. Can be superiority, equivalence or non-inferiority trials. All about comparing new drug to existing treatment.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
33
Q

What are Randomised Controlled Trials?

A

RCTs are considered the gold standard for the provision of evidence in medicine – it is hard to argue with a good RCT. In RCTs, participants are randomly allocated to one of two or more groups (at least one intervention group and one control group). They can have multiple intervention groups, for example different drugs, different doses, combos etc

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
34
Q

Randomised Controlled Trials: What is meant by Inclusion Criteria?

A

In RCTs, the inclusion criteria are usually quite strict, often including only people with the disease of interest and no other comorbidities in order to best show the effect of the drug

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
35
Q

Randomised Controlled Trials: What is meant by Randomisation?

A

Is a key point of the RCT process. Randomisation minimises the effects of confounding and bias

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
36
Q

Randomised Controlled Trials: What is meant by Blinding?

A

Either single or double blinded to minimise bias

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
37
Q

Randomised Controlled Trials: What is meant by Power and size?

A

Size of trial must have enough participants to determine a meaningful difference (be adequately powered). 80% power = 80% chance of detecting clinically important difference.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
38
Q

What do we get at the end of RCTs?

A

At the end of RCTs, the benefit is that we get easy to interpret, numerical data.

39
Q

What is the Placebo Effect?

What happens if the Placebo Effect isn’t accounted for?

How do we untangle the placebo effect from the real effect?

A

The placebo effect is the physiological effect caused by the patient’s belief that the treatment will do something.

If not accounted for, the placebo effect can lead to a systematic deviation from the true pharmacological effect.

We therefore try to untangle the placebo effect from the real effect by not telling the participant which treatment they are receiving, a process called blinding.

40
Q

Without blinding what will be present?

A

Bias

41
Q

The general reason why we use placebo groups and do blinding is why?

A

The general reason why we use placebo groups and do blinding is to ensure the two groups are as similar to each other as possible (i.e. the only difference is that one has the intervention and the other doesn’t)

42
Q

What are the Types of Blinding?

A

Single Blinding

Double Blinding

Double Dummy

Sham Control/Treatment

Combination

43
Q

What is Single Blinding?

What can this lead to?

A

Participants don’t know what treatment they are getting, but the doctors do.

This could lead to some bias, as they could subconsciously treat the groups differently to each other, especially when asking subjective questions

44
Q

What is Double Blinding?

A

Both the participants and the doctors do not know which treatments the subjects are getting, removing almost all sources of bias. This is the gold standard of blinding.

45
Q

What is Double Dummy?

A

Used when the two treatments cannot be made identical, for example drug A and drug B where one is a tablet and the other a capsule. In this case, we use an individual placebo for each drug, such that everyone will receive a drug and a placebo

46
Q

What is Sham control/treatment?

A

When a fake treatment, procedure, device or monitoring is used in comparison to the legit one. An example is a fake surgery, where the patient is put under and receives an incision etc. but does not get the actual surgery performed. There are ethical issues here, but we can balance the risk of the fake surgery with the greater good.

47
Q

What is an Example of a Combination of Blinding?

A

An example combining the two is a comparison between rivaroxaban and warfarin. They are two different drugs, so a double dummy is required. Furthermore, rivaroxaban does not require INR monitoring, so a sham control is also required

48
Q

When is blinding not used?

A

Blinding is not always used, as there are some cases where it may be difficult to maintain the blind.

An example is where the drug has a characteristic effect, such as hypotension (dizziness etc.), meaning the patient may be able to work out what treatment they received.

Another example would be oral vs IV, since giving a placebo IV could pose ethical issues (risk of infection, reactions etc.)

Another time blinding is not used is when the outcome of the trial is very objective and obvious, such as heart attack or tumour size.

49
Q

What are Limitations of RCTs?

A

Very expensive and labour intensive

As a result, sample size and duration are limited compared to the real world – may need extrapolation

Can only detect common side effects due to small size and duration, not rare effects. RCTs are mainly used for estimating therapeutic effect

Blinding is not always possible or foolproof in RCTs

Artificial study group (healthy volunteers), which reduces external validity

Can only ethically test something that has some evidence of benefit (through Phase I/II trials) – cannot just hop straight into a RCT

People might just choose not to follow the instructions

50
Q

Briefly describe Protocol Violations

A

Not all trial participants follow the full protocol. Some may change to the other treatment, while some will straight up stop their treatment due to side effects etc., or even some may be moved over to the other group by the investigators

51
Q

What is Censoring?

A

Censoring is the process of removing a subject from the trial, but not counting them has having achieved the outcome (for example mortality %)

It is used to overcome Protocol Violations

52
Q

Protocol Violations and Censoring: What Types of analyses can we perform to account for these people that drop out of trials?

A

Intention to Treat (ITT) Analysis, Per Protocol Analysis

53
Q

Protocol Violations and Censoring: Types of analyses we can perform to account for these people that drop out of trials - What is Intention to Treat (ITT) Analysis?

A

Still keep them in the analysis, even if they were non-compliant.

This is because randomisation only holds if everyone is analysed according to their original group they were randomly assigned to, and we need to keep randomisation to reduce bias

54
Q

Protocol Violations and Censoring: Types of analyses we can perform to account for these people that drop out of trials - What is Per Protocol Analysis?

A

Only analyse those who followed the protocol. The issue with this is that these people may have better outcomes than the types of people dropping out, leading to erroneous conclusions

55
Q

What is a Subgroup Analyses?

A

Subgroup analyses are when the effect of an intervention is investigated across sub-populations within the study groups.

These are intended to show whether or not certain subgroups are affected differently by the intervention

56
Q

Subgroup Analyses: What is treatment effect modification?

A

If the results of a subgroup analysis tell us that one subgroup benefits from the intervention more than another, then it is said to be treatment effect modification

57
Q

What is the Aim of subgroup analysis?

A

To detect treatment effect modification (i.e. different effects in different subgroups)

58
Q

It is often difficult to demonstrate a TEM, as the trial is not powered enough to do so, so any differences found are often not statistically significant (even if they may appear large). To find out if the differences found are statistically significant and hence if there actually is treatment effect modification, what can we do?

A

We need to see if the confidence intervals of the subgroups overlap, as well as with a test for heterogeneity

59
Q

How do we know if there is a TEM?

A

If the CIs do not overlap and p<0.05, then there is treatment effect modification (i.e. treatment has different effects in different subgroups). If these are not satisfied, then we can only say that the treatment has the same effect in both

60
Q

What is meant by Multiple Testing in Subgroup Analyses?

A

We can keep coming up with new subgroups to compare to each other, called multiple testing

61
Q

What is the Issue with Multiple Testing?

A

The issue with multiple testing and therefore having multiple subgroups is that eventually you will see a difference (TEM) between these groups, however this TEM will only be due to random chance

The more tests you do and groups you compare, eventually you will come across a false positive

62
Q

How do we Avoid the Issues with Subgroup Analyses?

A

Pre-specify subgroups (e.g. if you think it’ll be better in men before you start testing, then pre-specify the subgroups to be men and females)

Adjusting the p-value to the number of subgroups you have (i.e. if you have many subgroups, may want to lower p-value to reduce chance of seeing a difference that is caused by random error)

63
Q

What are Advantages of RCTs?

A

Bias is reduced through randomisation and blinding

Clear quantitative and comparable outcomes

Easiest to interpret results

64
Q

What are Disadvantages of RCTs?

A

Extremely expensive and logistically difficult

Artificial (groups tend to favour healthy individuals not reflective of general population)

Potential ethical objections

65
Q

What are Systematic Reviews?

A

Systematic reviews (SRs) are a process where the intention is to find and evaluate all available evidence related to a particular question. By evaluating all the available evidence and compiling them into a single review, based around a specific question, we create the best evidence possible

66
Q

What do SRs bring together?

A

Bring together studies performed by different people, in different settings, in different countries, for different lengths of time to look at different outcomes. This produces vast differences in the participants, outcomes and interventions between the studies, which is called heterogeneity

67
Q

What is Heterogeneity?

A

Heterogeneity refers to differences between the studies used in an SR in terms of the participants, outcomes and interventions involved

68
Q

What is the Process of Systematic Review?

A

1) Frame the question – start with a clear question you want to answer
2) Define inclusion/exclusion criteria for the studies – leaving out certain studies
3) Search for ALL primary studies fitting the above criteria
4) Critically appraise all studies on their quality – only include good studies in the review
5) Systematic approach to summarise the evidence and interpret the findings of all the studies

69
Q

What are the 2 Types of Heterogeneity?

A

Clinical and Methodological Heterogeneity

70
Q

What is Clinical Heterogeneity?

A

Clinical differences in the studies to do with the participants, interventions and outcomes e.g. age, sex, study location, dose of intervention

71
Q

What is Methodological Heterogeneity?

A

Study design, duration, quality, analysis

72
Q

What is a Meta-Analysis?

A

Meta-analysis is a technique used in systematic reviews to better estimate the size of a treatment effect by pooling the results of the different studies together

73
Q

How is a Meta-analysis conducted?

A

A meta-analysis is done using the studies used in the systematic review, so therefore it cannot be performed without the systematic review having occurred in the first place

74
Q

What is the End Result of a Meta-Analysis?

A

The summary estimate found at the end is the weighted average of the treatment effect estimates in the studies identified. Studies are weighted on their precision (i.e. their sample size)

75
Q

What are Meta-Analysis Useful for?

A

Meta-analyses are useful for improving the precision of the estimated treatment effect size by using all available data. Furthermore, by pooling so many studies together, we have a larger effective sample size, meaning small differences may be found. They may also identify heterogeneity in the treatment effect and the potential causes for this

76
Q

How are Meta-analysis summarised?

A

These analyses are summarised using a Forrest Gump Plot

77
Q

When does Statistical heterogeneity occur?

A

Occurs when the study results being compared in a systematic review appear to be very different. Study results will always differ due to random error; however, if the differences between studies are large and consistent and cannot be explained by random error, then the effect size is truly different between studies, indicating the presence of a TEM

78
Q

If we find that there is statistically significant heterogeneity amongst the studies, what can we use?

A

Then we can use meta-regressions to find the reason for this heterogeneity

79
Q

We can also perform a meta-analysis for subgroups of different studies to help identify what?

A

The size of the treatment effect modification (i.e. how big of a difference there is between groups)

This allows us to see what groups the treatment works better in, and to what extent.

80
Q

In a Meta Analysis, by pooling the subgroups of multiple studies, what do we increase?

A

We increase the chance of finding statistically significant subgroups

(helps to find heterogeneity)

81
Q

What are Limitations of Meta-Analysis?

A

Poor quality of data in primary studies can make meta-analysis worse

Studies selected may not be sufficiently comparable to each other

If there is lots of heterogeneity, is it legitimate to combine the results? Or will it just make things worse

Publication bias –studies showing an effect are more likely to be published in the meta-analysis, creating a bias towards the positive effect.

82
Q

How can we detect Publication Bias?

A

We can detect if there is publication bias in a meta-analysis using funnel plots

83
Q

When is Publication Bias detected?

A

If a funnel plot were to be created and the section of smaller studies with smaller effect sizes (or even negative effect sizes) was missing, this would indicate that these studies were intentionally left out of the meta-analysis in an attempt to make the result look better, hence indicating publication bias

84
Q

What is a Network Meta-Analysis?

When is it used?

A

There may be many RCTs using a number of different drugs to treat the disease you are interested in. However, you notice there are no trials directly comparing the two drugs you are interested in

Network meta-analyses are used to compare treatments via their effects with a common comparator (usually placebo).

85
Q

Usually the best study designs are what?

A

Prospective, Experimental, Controlled, Randomised, Large and Blinded

86
Q

What is Evidence Based Practice?

A

Integrating the best research evidence with clinical expertise and patient values to achieve the best possible patient management.

87
Q

What are the Steps in Evidence Based Practice?

A

1) Formulate a question
2) Review the evidence
3) Critically appraise the evidence
4) Apply the evidence – integrate the results with patient values and clinical experience
5) Evaluate the effectiveness and efficiency of the entire process.

88
Q

Evidence Based Practice: What is the Process we used to Formulate a Question?

A

P = Participants or Population


I = Intervention


C = Comparator or Control

O = Outcome

89
Q

Evidence Based Practice: How do we Review the Evidence?

A

When reviewing the evidence, we generally start from the bottom and work our way up. We begin with tertiary sources (sources that overview primary and secondary sources e.g. AMH), then secondary sources (interpretations of primary sources, e.g. systematic reviews and meta-analysis), then finally primary sources (the original material, e.g. RCTs themselves).

90
Q

Evidence Based Practice: What is meant by Critical Appraisal?

A

Once we have identified our articles, we need to critically appraise them and find if they have good internal validity. Studies with good internal validity are those with minimal bias and that utilise randomisation, blinding etc.

91
Q

Evidence Based Practice: Critical Appraisal - What questions do we ask to determine Internal Validity?

A

To check for internal validity, it is good to ask “how similar are the comparison groups?” and “do you really believe there is a difference in the intervention population?” If you answer “very similar” and “yes”, then there probably is high IV

92
Q

Evidence Based Practice: Apply the Evidence: How do you determine External Validity?

A

Often, RCTs have artificial study groups using only healthy volunteers, meaning worse results are seen in once the drug reaches the real world. Therefore, to determine external validity, we need to ask questions like “do you think you will see this same difference in the real world?”

93
Q

Evidence Based Practice: Apply the Evidence: What does External Validity Focus On?

A

Determining external validity focuses on how similar the study population is compared to the individuals that you will apply the results to (i.e. the ones receiving the treatment in the real world), or in other words how different is the study group to the real world. Is there bias in applying the results from the study group to the group of interest?

94
Q

If a study doesn’t have good Internal Validity, what about External Validity?

A

If a study doesn’t have good internal validity and reproducibility, then external validity doesn’t matter