Week 5

Question 1

Why do we need to understand advanced statistics in EBP?

Answer 1

• To effectively implement EBP and enhance patient care
  
  • Statistical proficiency supports informed clinical decision-making by distinguishing between random variations and meaningful patterns
  • It enables professionals to critically evaluate research, ensuring high-quality evidence informs practice
  • Understanding statistics also enhances patient safety, treatment effectiveness and cost-efficiency by guiding the assessment of interventions
  • It aids in quality assessment and public health planning, helping professionals generalize research findings to broad populations
  • Statistical literacy is essential for integrating new technologies and analytical methods into practice
  • It fosters professional growth and critical thinking, empowering healthcare professionals to communicate EBP insights to patients, enhancing shared decision-making
    Advanced statistical knowledge is fundamental for delivering high-quality, effective and patient-centred health care while driving innovation and improvements

Question 2

Key statistical tools

Answer 2

Regression analysis, Meta-Analysis, Forest Plots, Funnel Plots,

Question 3

What is meta-analysis

Answer 3

Meta-analysis:
- A statistical method that combines results from multiple studies to produce an overall finding
- Often accompany systematic reviews to increase statistical power by pooling data
It matters as rather than relying on one study meta-analysis gives a more robust estimate of an effect

Question 4

What is regression analysis

Answer 4

• A family of techniques for examining relationships between an outcome (dependent variable) and one or more predictors (independent variables).
  
  • e.g. regression can tell us how strongly patients therapy intensity predicts their recovery outcomes
    It matters as regression is widely used in health research to adjust for confounders and identify significant predictors

Question 5

What are forest plots

Answer 5

• A graphical representation of a meta-analysis
  
  • Each study is shown as a point estimate (usually square) with a horizontal line for its confidence interval and an overall summary effect is shown as a diamond
    It matters as they quickly let us see individual study results, their variability and the combined outcome in a single picture

Question 6

What are funnel plots

Answer 6

• A scatterplot used to check for publication bias in meta-analyses
  
  • It plots each study’s effect size against its precision
    It matters as an asymmetrical funnel plot may indicate smaller or negative studies are missing (possible bias) whereas a symmetrical funnel suggest a lower risk of publication bias

Question 7

Statistical significance

Answer 7

Statistical significance:
- A result unlikely to be due to chance along, typically determined by a p-value below a threshold (e.g. <0.05)
- This tells us an effect or difference probably exists in the sample data
- e.g. if a study reports that a new speech therapy technique improved language scores p=0.03, it means there’s only a 3% probability that this improvement was due to chance
Statistical significance reflects the influence of chance on the outcome

Question 8

Clinical significance

Answer 8

• The real-world importance or size of the effect
  
  • It asks: is the magnitude of the chance big enough to matter for patients or practice?
  • A result can be statistically significant yet so small it has little practical impact e.g. a trial finds a statistically significant difference in ROM after a new physiotherapy technique (p=0.047), but the actual improvement is only 1.1 degrees.
  • Clinically 1.1 degrees might not justify changing practice or investing in the new technique
    Always consider whether an effect would make a noticeable improvement in patient outcomes or decisions

Question 9

What is regression analysis

Answer 9

• Powerful for quantifying relationships between variables, we have:
  
  • Dependant variable (outcome): The health outcome or measure we want to predict or explain (e.g. patient’s improvement score, probability of hospital readmission)
    Independent variable (s) (predictors): Factors that might influence the outcome (e.g. number of therapy sessions, patient age, baseline severity)

Question 10

What is linear regression

Answer 10

• Used when the outcome is continuous (e.g. pain scale, ROM)
  
  • It fits in a straight line through the data
  • Outcome = a+b*(predictor) + error
  • The coefficient b tells us how much the outcome changes for a one-unit change in the predictor, holding other factors constant
    e.g. linear regression might find that each additional physical therapy session per week reduces pain score by 0.5 points on a 10-point scale, if all else is equal (with a certain confidence in that estimate)

Question 11

What is logistic regression

Answer 11

• Used when the outcome is binary (yes/no etc.)
  
  • Instead of a straight line, it predicts the log-odds of the outcome
  • Results are often expressed as odds ratios
  • e.g. a logistic regression could show that using a particular splint makes patients 2x more likely to avoid surgery (odds ration = 2) after controlling for injury severity
  • Tells us how predictors affect the odds of an outcome

Question 12

What is multiple regression

Answer 12

• Thes means regression with more than one predictor
  
  • In practice health outcomes are rarely caused by a single factor, so we include multiple variables in the model (e.g. both therapy sessions and patient age and initial status predicting recovery)
  • Allows us to adjust for confounders - those extra variables that might also influence the outcome
  • By adjusting for confounders, we isolate the effect of the main predictor of interest
    e.g. we might find therapy intensity predicts better mobility outcomes even after adjusting for patient age and baseline mobility

Question 13

What is a note for multiple regression

Answer 13

Note: adjusting for confounders means including factors in the model that could distort the main relationship. e.g. older patients tend to have few therapy sessions and also slower recovery. Age is a confounder for the effect of sessions on recovery. A multiple regression can adjust for age so that we can see the true contribution of therapy sessions on recovery independent of age

Question 14

How is regression used in allied health

Answer 14

• Interested in factors that predict rehabilitation outcomes after a stroke
  
  • Collect data from 100 patients including baseline walking speed, age, number of therapy hours
  • You run a multiple linear regression with walking speed improvement (m/s) as the outcome and the three factors as predictors
  • Regression results: Therapy hours per week have a positive coefficient (more hours is better improvement), meaning it’s a significant predictor of improvement
  • Baseline speed has a negative coefficient (patients who started better improve slightly less) and Age’s coefficient is near zero and not significant (age didn’t effect rehab improvement)
  • Interpretation: Holding age and baseline status constant , each additional hour of therapy per week is associated with 0.05 m/s increase in walking speed after rehab which is statistically significant
  • Baseline speed matters (those with very low initial speed had more to gain) while age did not show an effect

Question 15

Real world example of regression

Answer 15

• A recent study on stroke rehabilitation used regression to predict patient outcomes
  They found that initial impairment level and intensity of therapy were significant predictors of recovery, even after accounting for age and comorbidities. Helps clinicians identify which factors to focus on (e.g. ensuring patients get sufficient therapy intensity could improve outcomes)

Question 16

What is a forest plot

Answer 16

• Graph displaying the results of multiple studies side-by-side usually as part of a meta-analysis
  
  • Lets you instantly visualise the range of findings and the overall combined effect
    Each individual study in a meta-analysis is represented by a line in a plot and the pooled result is at the bottom

Question 17

Forest plot: Study names

Answer 17

Each row corresponds to one study (often identified by the firs author or year) e.g. Smith et al., 2018

Question 18

Forest plot: Effect size

Answer 18

• Each study’s result is shown as a square centred on its effect estimate (e.g. an odds ration or mean difference) with a horizontal line through it representing the CI (usually 95% CI).
  
  • If the line crosses the vertical ‘no effect’ line, that study’s results is not statistically significant (at the 95% confidence level)
    Large studies typically have narrower CI lines (more precise estimates) and smaller studies have a wider CI

Question 19

Forest plots: Weights

Answer 19

• The size of the square often indicates the weight of the study in the meta-analysis (larger squares = study contributed more to the overall result, usually because it had more participants or less variance
  Thus a big trial might have a big square, a tiny pilot study a tiny square

Question 20

Forest plot: Line of no effect

Answer 20

• A vertical line down the plot indicates the null effect (e.g. an odds ration of 1 or a mean difference of 0)
  
  • If a study’s CI crosses this line, its result isn’t statistically significant on its own
    The overall effect is significant if its summary diamond does not touch this line

Question 21

Forest plot: Overall summary (diamond)

Answer 21

• At the bottom a diamond shape represents the pooled result of all studies combines
  
  • The centre of the diamond is the combined effect estimate, and its width is the confidence interval
    If the diamond sits entirely to one side of the no-effect line (not crossing it), the meta-analysis indicates a statistically significant overall effect

Question 22

Forest plot: Heterogeneity

Answer 22

• Often a corner of the plot or a footnote will report an I2 statistic and possibly a chi-square (Q) test for heterogeneity
  
  • I2 (%) quantifies how much variability in results is due to differences between studies rather than chance
  • e.g. I2= 0% means all studies found essentially the same effect (no heterogeneity) whereas I2= 75%, would indicate high variability in results between studies
  • We interpret I2 roughly as 0-25% = 0-25% = -0.25 low heterogeneity, 50% = 50% moderate, 75%+ high heterogeneity
    High heterogeneity suggests the studies results differ substantially which can affect how confident we are in the combined result

Question 23

Guided interpretation

Answer 23

1. Overall direction: are most of the study squares on one side of the line of no effect (if yes, that tells you the general trend of results)
   
   2. Significance of each study: Do any of the horizontal lines not cross the line of no effect (those studies have statistically significant findings on their own)
   3. Most influential study: Which study has the largest square (this study has the greatest weight. Perhaps it had the largest sample size. Its results will pull the combined results more strongly)
   4. Pooled result: Look at the diamond. Is it left or right of the line, or overlapping. What does it say about the overall effect and its significance
      Heterogeneity: If an I2 value is given, is it low, moderate or high? (how consistent were the study results? If high, consider what might differ between studies e.g. different patient characteristics or protocols)

Question 24

What is a funnel plot and why use it?

Answer 24

• When we conduct a meta-analysis, we rely on having all relevant studies
  
  • But what if some studies were never published, especially those with negative or inconclusive results
  • Publication bias can skew evidence - typically studies with positive findings are more likely to be published than those with null results
    A funnel plot is a tool to detect such bias

Question 25

A funnel plot is essentially a scatter plot of each study in a meta-analysis

Answer 25

- The x-axis is the study's result (often the effect size or outcome measure) - The y-axis is a measure of the study's size or precision (often the inverse of the variance or standard error, or sometimes sample size) Larger, more precise studies appear toward the top of the plot, smaller studies appear toward the bottom

Question 26

Types of funnels

Answer 26

- In the absence of bias, we expect the plot to resemble an inverted funnel (or pyramid) that is roughly symmetrical Why: because big studies (at the top) will cluster near the true effect size and small studies (bottom) will scatter more widely on both sides of the true effect (purely by chance some small studies overestimate, some underestimate) this forms a symmetrical funnel shape

Question 27

Symmetrical funnel no bias

Answer 27

- Large studies at the top cluster around the average effect - As you go down (smaller studies) points spread out but evenly on both sides of the average This symmetry suggests no strong publication bias - were seeing both positives and negatives in small studies

Question 28

Asymmetrical funnel (potential bias)

Answer 28

- If the funnel is skewed (many small studies on one side of the average but not on the other side) it may indicate that some studies are missing - likely the missing ones are those that would have fallen the blank side of the plot - Often its negative or unfavourable results that go missing - e.g. if no small studies with no effect were published, you'd only see small studies with big effects making one side of the funnel sparse - An asymmetric funnel plot raises suspicion of publication bias or other small-study effects Another cause asymmetry can be true heterogeneity e.g. small studies being conducted differently than large ones, yielding different effects, but either way, asymmetry = reason to investigate

Question 29

Example of a funnel plot

Answer 29

- Less studies on the left side of the vertical line and the spread is uneven - This asymmetry implies that perhaps studies with outcomes on that missing side were not published or not found - Pattern suggests potential publication bias or systematic differences in smaller studies Further statistical tests could confirm bias but visually it already prompts caution

Question 30

Interpreting a funnel plot

Answer 30

1. Check symmetry: Do the points form a nice roughly symmetric triangle or is one side sparsely population 2. Look at the top: The largest studies at the top should be near the true effect (around the vertical line). Even if the top appears skewed, that might indicate issues beyond publication bias (maybe different populations) 3. Consider implications: If asymmetry is evident, the reviewers might conclude that the meta-analysis could be overestimating the effect (since perhaps missing studies with smaller or negative effects) they may perform adjustments or at least discuss this limitation. If the funnel is symmetric, it increases confidence that the meta-analysis is not missing a bunch of unfavourable studies 4. Remember: A perfectly symmetric funnel is deal but not always seen. Moderate asymmetry could occur by chance especially with few studies. We typically look at funnel plots when there are around 10 or more studies, fewer than that the plot isn't very reliable for bias detection

Question 31

Why do clinical trials matter in EBP

Answer 31

- Role will include delivering interventions that support clients health and wellbeing - But how do we know which interventions are effective? This is where clinical trials come in, especially randomized control trials (RCT's) which are considered one of the most reliable ways to test whether an intervention works

Question 32

Understanding the basics of clinical trials

Answer 32

- A type of research study that tests the effects of a health intervention - such as a new therapy technique, group program, medication or even preventative strategy - The goal is to determine whether the intervention improves health outcomes compared to another approach - RCT's use randomization: participants are randomly allocated to different groups e.g. intervention and control group Because allocation is random, each group is likely to be similar at the beginning of the study, which make the results more trustworthy

Question 33

Why randomization matters

Answer 33

- Imagine you are evaluating a new group-based therapy for children with autism, if you just compared outcomes for the children who chose to attend the group with those who didn't, you might find differences- but those differences could be cause by many things such as parental motivation, severity of symptoms or family support, not necessarily the therapy itself - By randomly assigning participants, you reduce the chance that these other factors called confounding variables are influencing your result Makes sure that any differences in outcomes between the groups at the end of the trial are more likely to be due to the intervention itself. This strengthens what we call the internal validity of the study, that is how confidently we can say the intervention caused the effect

Question 34

Limitations and challenges

Answer 34

- Not always perfect or possible - Difficult to create a placebo version of a therapy - Challenged to cost, recruitment and ensuring the results apply to diverse populations Understanding how clinical trials work and how to interpret their findings is essential to EBP, allows you to critically evaluate the research behind the interventions you use, justify clinical decisions and contribute to improving the care and outcomes

Question 35

Reading and interpreting RCT

Answer 35

Reading and interpreting a RCT Critical appraisal: Taking a closer look at the design, methods and results of a study to decide whether you can trust the findings and apply to client group

Question 36

Features when reading an RCT: Who was studied

Answer 36

Population, ages, diagnoses, similarity to clients, external validity

Question 37

Features when reading an RCT: How were participants assigned to groups

Answer 37

- Was the process random - how was randomization carried out, a good RCT will explain its method. - Randomization helps ensure that both groups are similar at the start of the trial, which means any differences at the end are more likely to be cause by the intervention. If randomization wasn't used, or wasn't clearly described, the results may be biased

Question 38

Features when reading an RCT: Was the trial blinded

Answer 38

Participants and sometimes researchers don’t know which group people are in. This reduces the chance that expectations or behaviour changes will influence the results e.g. if a client knows they're receiving the new treatment, they might try harder or expect it to work, which could influence the outcome (placebo effect)( not all AH interventions can be blinded)

Question 39

Features when reading an RCT: What was the intervention and what was it compared to

Answer 39

A strong RCT clearly describes both the intervention and control condition, the intervention should be something that could realistically be delivered in clinical practice. The control group might receive no treatment, usual care or an alternative intervention

Question 40

Features when reading an RCT: What outcomes were measured

Answer 40

- The study should explain what outcomes it was trying to influence - like changes in function, symptoms, participation or quality of life - These should be outcomes that matter to clients and clinicians How outcomes were measures. Were tools valid and reliable, were they appropriate for the population studied? For example, was the standardised communication scale used in a speech pathology trial? Was it culturally appropriate

Question 41

Features when reading an RCT: How large and how meaningful were results

Answer 41

- Is this a meaningful change e.g. a statistically significant result might show that an intervention reduced anxiety scores by 2 points, but if that change isn't noticeable to clients or doesn't affect daily living it might not be clinically important Effect sizes can give you more information about how big the difference really was

Question 42

Features when reading an RCT: Were there any sources of bias or missing data

Answer 42

- Check if the researchers discussed limitations, like dropouts, unequal group sizes, lack of blinding - These can introduce bias - e.g. if many participants in one group dropped out, the final comparison might be skewed (attrition bias) - A well-conducted RCT will use strategies like intention-to-treat analysis, which includes all participants in the group they were originally assigned to, even if they didn’t complete the study Helps maintain the benefits of randomization

Question 43

Features when reading an RCT: Do the authors conclusions match the results

Answer 43

- Look at how the authors interpret their findings - Do their conclusions reflect the actual results, or do they overstate them? - A good discussion will acknowledge both the strengths and the limitations of the study and make reasonable claims about what the intervention might achieve

Question 44

What is multiple regression

Answer 44

Multiple regression: Allows us to control for other things so we can focus on one key factor

Question 45

Forest plots:

Answer 45

- Have a vertical line (line of no effect), with no interventions between to groups - If the horizontal line doesn’t cross the vertical there is statistical significance - All are different but interpreted the same - Box and whisker plot for them all - Diamond for total - Forest plots summarize studies - Size of the square: Sample size, the bigger the box the bigger the sample size - Middle line: Line of effect: Studies either cross or don’t cross the line. Means whether the studies have show significant effects or not - Diamond: summary of the effect size, overall effects/result of all the studies - If the diamond touches the vertical line, the whole result was not statistically significant - Left side: less interest in the treatment group - Right side: more episodes of outcome in the treatment group Don’t want the diamond to cross the line: Overall result, hasn't crossed the line

Question 46

Funnel plots: Explained

Answer 46

- Looking for publication bias - Symmetry or asymmetry of the dots - Dotted lines represent a confidence interval - Symmetry: No/less likely of publication bias - B has asymmetry is missing dots - Might be more skewed as: Data or studies could be missing, might have small study effects, which skew funnel plot towards one side, some could be unpublished, missing from analysis What we want to see whether we can trust the whole analysis in terms of publication bias

Question 47

What is the p value and the null hypothesis

Answer 47

- The assumption that there is no effect or difference or no association in the population - Small p value suggests the observed effect is unlikely to be due to chance leading us to question the null hypothesis - P less than 0.05: Statistically significant P more than 0.05 not statistically significant: The effect could be due to chance

Question 48

What is a point estimate

Answer 48

- A calculation where a sample statistic is used to estimate or approximate an unknown population parameter - e.g. the average height of a random sample can be used to estimate the average height of a larger population

Question 49

What are confidence intervals

Answer 49

- A range either side of the point estimate that tells you how much the point estimate may vary in the population - Sometimes described as a margin of error Confidence limits are simply the extreme ends of the CI (the highest and lowest values of the interval)

Question 50

Relationship between the p value and CI

Answer 50

- If the CI does not cross the null effect value (e.g. 1 for odds ration or 0 for mean difference) p is less than 0.05 - If the CI crosses the null effect value p is more than 0.05

Question 51

What is odds ratio and the diamond in a forest plot

Answer 51

The odds of an event is the number of cases who have experienced the event of interest divided by the number of those who do not experience the event of interest What is the diamond in a forest plot? Each side of the horizontal line is what the study favours, control or intervention