Lecture 10: effect evaluation

Question 1

What is evaluation?

Answer 1

Determines the degree to which the
campaign reaches its objectives: it helps planners and scholars understand how and why a particular campaign worked and it provides information relevant for planning
future activities (McKee, 2004)

> Making an evaluative judgment
A critical judgment about strong and weak points of campaign and how this can be improved
Did campaign reach objectives?

Question 2

Why evaluate campaigns?

Answer 2

Knowledge
>Determine / understand campaign effects
>Figure out ‘best practices’ / effective design-principles
>To improve: learn from successes and failures
>Quality (future) meta-analyses depend on quality of individual campaign evaluations (reliable estimate effect sizes)

But also:
>Justify costs/resources used
>Determine return on investment
>Get public/political support
>Show successes to secure future funding

Question 3

2 other important concepts related to effect evaluation

Answer 3

> Efficacy

>Effectiveness

Question 4

Efficacy

Answer 4

> Campaign evaluated under optimal conditions
Lab research
High internal validity
Low external validity

Usually before campaign launch

Question 5

Effectiveness

Answer 5

> Campaign evaluation under ‘real life’ conditions
Field research
Difficult (and expensive)

Question 6

Golden standard

Answer 6

> Randomized Controlled Trial (RCT)
Intervention group vs. control group
Random assignment to groups
Participant and researcher do not know who gets what treatment

Question 7

Validity

Answer 7

External validity
>Can we generalize the results to other populations, situations and
time periods?

Internal validity
>Can we really attribute results to the intervention/campaign?

Question 8

Sample

Answer 8

Different independent samples (cross-sectional)?
>Every time different participants
>+ less social desirability
>- long period: changing composition population

Panel design (longitudinal)?
>Every time the same participants
>+ within-participants, changes over time
>- non-response bias due to dropout
>- social desirability (participant wants to report progress)