Evaluation design I Flashcards Preview

ESS3705 Sport, Physical Activity and Health > Evaluation design I > Flashcards

Flashcards in Evaluation design I Deck (55)
Loading flashcards...
1

stages of evaluation

formative

process

outcome

The main purpose of evaluation design is to be as confidence as poss that any observed changes were cause by intervention, rather than by chance/other unknown factors

2

formative eval

Before intervention

Acceptability and feasibility of intervention

Mainly qual, e.g. focus groups, in depth interviews

3

formative eval research

Penn et al. (2018)

4

Penn et al. (2018)

NHS diabetes prevention programme

eval - qual research

behav interventions

specification reflected evidence - framework for service provision

provides ev based behav intervention for prevention of T2D in high risk adults

5

process eval

Measures how intervention delivered and received

Mixed qual and quan

Done along the way - make alterations if necessary

6

process eval research

Sanchez et al. (2017)

7

Sanchez et al. (2017)

improve understanding of underlying mechanisms that may impact results

prescribe healthy life intervention

moderate-->good performance on adoption, reach and implementation

8

outcome eval

Measures whether intervention achieved objectives

Mainly quan

9

outcome eval research

Ebert et al. (2018)

10

Ebert et al. (2018)

internet and mobile stress management intervention and RCT

intervention v control

int = 7 sessions of problem solving and emotion regulation techniques

baseline v 6 months

cost-effective and lead to cost savings

11

stages of evaluation research

Dehar et al. (1993)

Nutbeam (1998)

12

Dehar et al. (1993)

formative - develop and improve programmes at an early stage

process - info on programme implementation, interpretation of outcomes and guiding future research

13

Nutbeam (1998)

issues with definition and measurement of outcomes and use of eval methods

most powerful interventions = LT

technical problems from scientific rigour and advantages of less well defined content

combine and quan and qual

evals tailored to intervention

14

cause and effect

Want to determine whether a cause-effect r'ship exists between intervention and outcome

15

logic of causal inference

Under what conditions may we infer that a change in the DV (PA) was really caused by IV (intervention) and not by something else (envs etc.)

What are some of the most plausible rival explanations, and how do we rule them out?

16

Logic of causal inference research

Rothman and Greenland (2004)

Hill (1975)

17

Rothman and Greenland (2004)

causality often debated

more general conceptual model required

18

Hill (1975)

https://journals.sagepub.com/doi/pdf/10.1177/003591576505800503)

19

criteria for inferring causality

temporal r'ship

plausibility

strength of association

dose-response r'ship

reversibility

20

temporal r'ship

Cause (intervention)must precede effect (increase in PA)

21

plausibility

Association plausible, and more likely to be causal, if consistent with other knowledge

E.g. reasonable to expect people who receive exercise intervention will increase PA by greater amount than people without intervention

22

strength of association

Strong association, as measured by effect size or relative risk, more likely to be causal than weak association

23

dose-response r'ship

Occurs when changes in level of cause associated with changes in prevalence/incidence of effect

Greater adherence to exercise intervention (greater dose) the greater change in fitness

Incremental change in outcome, related to dose of intervention

1 intervention - adherence = linked to dose

24

reversibility

When removal of cause (intervention) results in return to baseline in outcome (PA), likelihood of association being causal strengthened

25

Internal validity

High internal val means that diffs observed between groups related to intervention tested in trial

Means for example, that change in PA in study popn attributed to intervention and not to other factors, such as age, sex/social status

IV has to be only thing influencing DV

Extent to which you're able to say that no other variables, apart from the IV, caused a change in the DV

Control and treatment group - treat them the same - see whether other variables influencing results

26

Internal validity research

Halperin et al. (2015)

27

Halperin et al. (2015)

internal val - degree of control exerted over confounds to reduce alternative explanations for results

e.g. controlling ex protocols, prior training, nutritional intake, age, gender etc.

less well controlled: instructions given, verbal encouragement, no. of observers and mental fatigue

28

External validity

Described extent to which results of experiment of intervention can be generalised to target/general popn

The extent to which the results of a study are generalisable to other situs/groups

Good sampling needed
- Representative of wider popn

- Large enough to have adequate power

- Exclusion criteria should relate to Q of interest

29

External validity research

Leviton (2017)

30

Leviton (2017)

need greater focus on external val

goal is applicability

methods
- better description of int
- combining of stat tools and logic to draw inferences about samples
- sharper definition of theory
- more systematic consultation of practitioners