Lecture 5 (Quantification, Forecasting, and Measuring Public Policies)

Question 1

Government metrics (Erkilla)

Answer 1

Quantifiable measures (number-based measures) used to evaluate, monitor, and improve policy outcomes (e.g. number of vaccinations distributed)

Question 2

The role of numbers in the construction of reality (Erkilla)

Answer 2

Numbers can be used to construct a desired reality by governments (or other powerful actors), to conceal their agenda as numbers are supposed to be neutral facts.

Question 3

Benchmarks (Erkilla)

Answer 3

A quantitative standard or criterion used to compare national policy performance against international standards

Question 4

Consequences of using benchmarks (Erkilla)

Answer 4

Structured pressure for countries to align with supra-national norms (e.g. agreed-upon EU-norms) and policy convergence (homogeneity) resulting from the fear to be critiqued or not receiving certain funds

Question 5

The ranking industry

Answer 5

Those actors that come up with quantitative knowledge, and in which type (e.g. universities, governmental organisations, NGO’s, etc.)

Question 6

Two types of quantitative knowledge

Answer 6

1. Objective indicators
2. Subjective indicators

Question 7

Objective indicators

Answer 7

Empirical facts independent of subjective interpretation (e.g. percentage of women with ADHD)

Question 8

Subjective indicators

Answer 8

Subjective assessments of situations (e.g. expert opinions)

Question 9

Framework to classify measurements into 6 aspects (6 aspects to consider when confronted with numbers)

Answer 9

1. Type of producer of the numbers
2. Purpose of the numbers
3. Scope covered by the data
4. Method of data collection
5. Presentation of the results
6. Publication strategy and visibility

Question 10

Type of producer of numbers (framework 1)

Answer 10

What are the motives and resources of the organisation producing this data? (e.g. public administration, universities, etc.)

Question 11

Purpose of the numbers (3) (framework 2)

Answer 11

1. Research/academic
2. Instrumental (for resource allocation/policy evaluation)
3. Benchmarking

Question 12

Scope covered by the data (framework 3)

Answer 12

Can be on a continuum/spectrum from particular specific phenomenon to a broader analytical phenomenon (often, to get to the broader analytical phenomenon, smaller phenomena are compiled)

Question 13

Method of data collection (framework 4)

Answer 13

Is the data collected via objective indicators or via subjective indicators?

Question 14

Presentation of results (framework 5)

Answer 14

Watch for if the presentation of the data may be misleading (e.g. small effects being exaggerated by cutting axes of graphs)

Question 15

Publication strategy and visibility (framework 6)

Answer 15

Think about at whom the data is directed, what group is supposed to do something with it?

Question 16

Outsourcing decision-making to number-based strategies (Saltelli & Di Fiore)

Answer 16

Governments tend to outsource decision-making (considering pro’s and con’s, etc.) to number-based strategies. They ‘let the numbers decide what is best’, thus outsourcing the decision-making responsibility to numbers. (kind of similar to how governments outsource public services to private companies to dodge responsibility)

Question 17

Luhmann’s 4 subsystems and their unique codes

Answer 17

1. Science (true/false)
2. Economy (profit/loss)
3. Media (news/no news)
4. Technology (functions/dysfunctions)

Question 18

Why it is not a good idea to combine subsystems in numbers acc. Luhmann

Answer 18

Combining the subsystems and their codes can undermine their autonomy, causing them to dominate and corrupt each other (e.g. economy corrupting science in regulatory capture).

Question 19

Two major concerns with quantification (Saltelli & Di Fiore)

Answer 19

1. Inequality being imbedded in government algorithms (e.g. toeslagenaffaire)
2. Poor modelling resulting in wrong or unjustified policies (e.g. reliance on wrongful models leading to dysfunctional policy responses neglecting the needs of certain groups)

Question 20

Ranking of universities and its consequences (Cathy O’Neal) (example of poor modelling resulting in wrong/unjustified policy choices)

Answer 20

Universities want to be as high as possible on the ranking lists. For example, this causes them to want to hire more good teachers, and lessen the number of students per teacher. This makes uni’s more selective in which students are allowed to follow their programs, and simultaneously the tuition fees will be increased. This will exclude certain groups from attending uni. These phenomena make the ranking list worthless, as all uni’s engage in similar practises.

This is a result of poor modelling (ranking uni’s assuming that means something) leading to unjustified policy choices (e.g. becoming more selective and raising tuition fees).

Question 21

Perverse quantification (Saltelli & Di Fiore)

Answer 21

Number-based decision-making assuming that numbers are neutral. This results in not taking into account possible consequences of decisions.

Question 22

Technology of hubris (hoogmoed) (Sheila Jasoff)

Answer 22

When one relies on the idea that numbers are neutral and objective, which results in overconfidence in them, underestimating uncertainty within those numbers. What the numbers say shall be done.

Question 23

Virtuous quantification

Answer 23

Focused on recognising that numbers still come with a certain degree of uncertainty, and that many factors have to be taken into account in decision-making).

Question 24

Technology of humility (Sheila Jasoff)

Answer 24

Focuses on reflecting the ambiguity in numbers and their multiple possible interpretations. Winners and losers of decisions about costs and benefits are identified.

Question 25

Ethics of quantification (Saltelli & Di Fiore)

Answer 25

A framework for understanding and evaluating numbers, metrics, models, and algorithms in public decision-making and policy-making

Question 26

Key elements of the ethics of quantification (4) (Saltelli & Di Fiore)

Answer 26

1. Explain the implications of model uncertainty (so no blind decision-making based on them) 2. Recognise that quantification is never neutral (models always reflects interests and biases) 3. Recognise numbers are ambiguous (there are still things you do not know --> tech. of humility) 4. Think about what the numbers are for: illumination vs. obfuscating

Question 27

Illuminaiton (Saltelli & Di Fiore)

Answer 27

When quantifications brings attention to previously invisible issues (e.g. death rates of illness)

Question 28

Obfuscation (Saltelli & Di Fiore)

Answer 28

When quantification strategically, or unintentionally, hides or downplays other relevant dimensions (e.g. focus on COVID death rates removing focus from consequences vulnerable economic groups)

Question 29

Selective use of obfuscating (Saltelli & Di Fiore)

Answer 29

Government and other powerful actors may use quantification to selectively illuminate aspects that can support their interests, whilst obfuscating those that do not

Question 30

The welfare state as a dependent variable (Clasen & Siegel)

Answer 30

The welfare state becomes a dependent variable when researchers study why welfare states differ from each other (so, the welfare state is influenced by some other variable like politics, economy, etc.)

Question 31

The welfare state as an independent variable (Clasen & Siegel)

Answer 31

The welfare state becomes an independent variable when researchers look at what effects the welfare states have on societies (the dependent variable)

Question 32

Path dependency (Clasen & Siegel)

Answer 32

Path dependency is central when researchers treat the welfare state as a dependent variable, and thus try to understand why welfare states differ. Path dependency is central to explaining this as past policy decisions and historical events constrain or enable future development of the welfare state.

Question 33

Stratification consequences (Clasen & Siegel)

Answer 33

Stratification consequences is central when researchers study the welfare state as an independent variable, and thus try to understand how a welfare state influences its society. Stratification consequences refersto how welfare states tend to reduce social stratification or reinforce it.

Question 34

The welfare state as an employer (Clasen & Siegel)

Answer 34

When studying the welfare state as an independent variable, and thus how it influences its society, looking at the role of the welfare state as an employer is central. The employment practises of the welfare state influence its labour market ((gender) inequality, commodification, decommodification, etc.)

Question 35

There are two aspects of studying the welfare state that must be adressed when comparing and analysing welfare state policies (Clasen & Siegel)

Answer 35

1. Operationalisation 2. Measurement of welfare state policies

Question 36

Operationalisation: how to conceptualise the welfare state so it can be studied (analysing welfare state) (Clasen & Siegel)

Answer 36

1. Resource allocation (e.g. expenditure patterns) 2. Rules and regulations (who gets what benefits under which conditions) 3. Distributional outcomes (studying real-world effects of welfare state policies, such as poverty rates)

Question 37

The problem with expenditures in operationalising the welfare state (Clasen & Siegel)

Answer 37

Without context, expenditures tell you relatively little - you do not know wether expenditure patterns reflect generosity, or addressing a problem via cash benefits, or if something are legitimate costs or just an ineffective policy.

Question 38

When operationalising the welfare state: Expenditure patterns can result from three factors

Answer 38

1. Generosity of benefits (amount of provided support by the welfare state, and the quality of the services) --> more benefits = higher expenditure 2. Access to benefits (the eligibility criteria for benefits) --> more eligible people = higher expenditure 3. Size of population of beneficiaries (the people that qualify for the benefit) --> higher population of beneficiaries = higher expenditure

Question 39

The practical tools and data used to compare information about welfare states in the measurement of welfare state policies

Answer 39

1. Administrative data 2. Total financial costs of the economy 3. Total costs to society