Working with Administrative Data

Question 1

Which of these is an example of administrative data

Text from a tweet
Information from a birth certificate
Demographic information collected during a baseline survey
Income information from tax records
None of the above

Answer 1

Both the information from a birth certificate and income information from tax records are data that are collected during the normal operations of a program and not primarily collected for specific research.

Question 2

Compared to survey data, administrative data are less susceptible to _______ bias because _______

Answer 2

recall bias,

because data are collected at the time of occurrence

Question 3

T/F: In the context of randomized evaluations, researchers can obtain administrative data from both public (i.e., governmental) and private institutions.

Answer 3

True

Both public and private institutions have provided individual-level administrative data to researchers for the purposes of randomized evaluations.

Question 4

In regards to administrative data, the _______ identified and sensitive the data that you are asking to be released, the _______ challenging it will be to get those data outside of the agency for research.

Answer 4

more, more

Question 5

When choosing identifiers for matching study data to administrative data, which of the following identifiers would be preferable to using an individual’s street address

Answer 5

A government-issued, unique identification number
Date of birth

because these are both numerical identifiers as opposed to identifiers comprised of letters and numbers

Question 6

The exact/deterministic matching strategy may lead to more ___________, while the fuzzy/probabilistic matching strategy may lead to more ___________.

Answer 6

False negatives, false positives

fuzzy and probabilistic matching strategies can account for minor discrepancies, but may lead to more false positives. On the other hand, exact and deterministic matching strategies do not account for minor discrepancies and might lead to more false negatives

Question 7

During the data matching process, the _______ file and the _______ file are combined to create the _______ file

Answer 7

identified finder, administrative data, de-identified analysis

Question 8

identified finder file

Answer 8

contains the identifiers of the study sample and a study ID. The study ID is a numeric ID that uniquely identifies each person in the study.

Question 9

administrative data file

Answer 9

The data provider has the administrative data file that contains identifiers and the outcome variables of interest to the research team.

Question 10

de-identified analysis file

Answer 10

The data flow process will determine how the identified finder file and the administrative data file are combined to create the de-identified analysis file

Question 11

In addition to the data provider, who should sign the Data Use Agreement (DUA)?

Answer 11

An official institutional representative

rather than an individual PI or staff member.

Question 12

If the research team never comes into contact with individuals in the study, they do not need to get IRB approval to use the individuals’ administrative data.

T/F

Answer 12

False

Even though the research team may never come into contact with the individuals whose information is included in the administrative data, it may still be necessary to complete the IRB process, even if just to confirm “exempt” status.

Question 13

Reporting bias

Answer 13

occurs when people have incentive to under- or over-report information.

Question 14

Why are administrative data useful?

Answer 14

The outcomes and metrics required for a study may already be tracked by a government or organization
• Available retrospectively
• Enable long-term follow-up
• Reduce logistical burden
• Include near census of relevant population
• Often cheaper than surveys

Question 15

How do administrative data minimize recall bias?

Answer 15

Data recorded at the time of occurrence– no memory

needed (e.g., banking records)

Question 16

How do administrative data minimize social desirability bias?

Answer 16

Non-self-reported data (e.g., arrest records)

Question 17

How do administrative data minimize Differential attrition and non-response bias

Answer 17

Near census of relevant population

Question 18

Identifiable vs. partially de-identified, de-identified

Answer 18

Identifiable - very easy to identify individuals

Partially de-identified - more difficult; but still possible especially with additional knowledge to piece together

De-identified - very difficult or impossible to identify

Question 19

Exact/Deterministic Matching

Answer 19

Minor discrepancies are not well accounted for
– E.g., typos in name, reversed day and month in DOB
• Some records are not identified as matches even though they may be (false negatives)

Question 20

Fuzzy/Probabilistic Matching

Answer 20

Accounts for the likelihood that identifiers may not align exactly to those in a data system
– E.g., SSN and last name match, DOB is off by a month…counts as a match

Question 21

Differential Coverage bias

Answer 21

Differential ability to link individuals to administrative records
• Treatment and control are differentially likely to appear in administrative records

Question 22

To address differential coverage bias

Answer 22

• Collect identifiers for linking during the baseline
survey
– To ensure that you are equally likely to be able to link
treatment and control individuals to their records
• Identify the data universe
– Which individuals are included in the data and which are excluded, and why?
– To ensure the intervention does not affect the likelihood of appearing in a data set

Question 23

Differential Reporting

Answer 23

• Likelihood of reporting outcome is correlated with treatment
– True value of the outcome may not differ between treatment and control, but due to the intervention, treatment group is more likely to report a certain outcome or appear in administrative records

Question 24

To address differential reporting

Answer 24

Identify how the intervention may affect the reporting of
outcomes
– Identify the context in which the data were collected
– Determine direction in which estimates are likely to be biased
– E.g., do number of doctor’s visits reflect severity of sickness or stronger connection to the health system?

Question 25

To address possibly inaccurate data

Answer 25

• Cross-reference with other sources to ensure accuracy
• Identify the data agency’s quality control protocol
• Choose indicators that are unlikely to be incorrectly reported
– Select variables that are straightforward and less susceptible to human error
– Request raw variables
• Communicate with program or implementing partner
responsible for collecting data
– Ask how and why data are collected

Question 26

Reporting Bias

Answer 26

• From an individual
– E.g., under-reporting income to qualify for a social welfare program
• From an administrative organization
– E.g., schools over-report attendance to meet
requirements

Question 27

To address reporting bia

Answer 27

• Identify the context in which the data were collected
– Were there incentives to misreport information?
• Choose variables that are not susceptible to bias
– E.g., hospital visit v. value of insurance claim