Process evaluation

Question 1

Definition of process evaluation

Answer 1

Moore et al 2015—-> Factors that shape theories, implementation and causal mechanisms

“Anything EXTERNAL to the intervention that may act as a barrier or facilitator to the intervention, or its effects”

Question 2

Process Evaluation 1

Answer 2

Description of intervention and causal assumptions

• What is it you want to do as part of the intervention?
• Evidence?
• How do you think this intervention will improve outcomes for participants?
• Consider this BEFORE conducting the intervention
• Bring together stakeholders for their opinions

Question 3

Process Evaluation 1: Evidence

Answer 3

CHErIsH Study ~ Choosing Healthy Eating for Infant Health

Evidence-based intervention- improve parental and infant feeding behaviours, and target healthcare professional (HCP) behaviours to support and sustain the delivery of the intervention

Two levels to intervention
– HCPs delivering the intervention
– Parents being provided with the intervention

Question 4

Process Evaluation 2

Answer 4

Borrelli (2011) – Implementation and what is delivered
Fidelity
intervention delivered as intended/per protocol?
Training given?

Detailed explanation of the rationale and methods for evaluating
Dose
Quantity implemented

Question 5

How is Process Evaluation 2 measured?

Answer 5

• Can be assessed by:
  – recording sessions, comparing what should be delivered (content of intervention manual) and what is actually delivered. BCT Taxonomy can be used here.

Intervention fidelity
* Degree to which the intervention is implemented as intended
* Confidence that the outcomes of the intervention are due to the intervention itself, rather than other factors.
* Related to
* Intensity of intervention
* Duration of intervention
* Delivery

Measured by number of sessions and type of therapy

Question 6

Process Evaluation 3

Answer 6

Mechanisms of impact, including mediators

• Key factor(s) leading to improvement for participants?
• differ amongst key stakeholders (participants, healthcare professionals)?
• Mediator – factor which helps to explain the relationship between treatment and outcome

Question 7

Process Evaluation 4

Answer 7

Outcome
* Outcome most important to:
– Patients?
– Healthcare professionals?
– Researchers?
– Other stakeholders?

Question 8

How to measure outcome?

Answer 8

age, gender, ethnicity, locality and any other factors

Question 9

Why is Process evaluation important?

Answer 9

Delivery of intervention—-Improvement in outcome

Delivered in the way it was supposed to be?
Therapeutic relationship between patient and HCP delivering the intervention the true driving factor

Potential for bias in outcome reporting?

Question 10

Why is Process evaluation important? 2

Answer 10

encourages us to think in-depth about the rationale for the intervention (theoretical background; evidence base)

allows us to learn more about the practical aspects of interventions that might impact on whether the intervention is successful or not

enables us to assess whether the intervention could be rolled out in a real-world setting

Question 11

What are the cons of Process Evaluation?

Answer 11

• Difficult for one intervention study to cover all areas – will often focus on one or two key areas, depending on the intervention, setting, etc.
• Mix of quantitative and qualitative methods