W6: Missing Data

Question 1

Name 2 reasons why missing data is problematic.

Answer 1

1. Results from non-missing data = biased
2. Causes loss of efficiency

Question 2

What is list-wise deletion?

Answer 2

Analysing only complete cases
E.g discard data in x and z because observation on y is missing

Question 3

What is MCAR (missing completely at random)?

Answer 3

Missingness completely independent of the estimate of our parameter(s) of interest
E.g dog is the missingness mechanism of homework with missing values regardless of value of homework

Question 4

Is there an empirical way to determine which missing data assumption is correct? (I.e MCAR / MAR / MNAR?)

Answer 4

No

Question 5

What is 1 way to assess missing data assumptions?

Answer 5

Sensitivity analysis: what happens to results by comparing imputed data and list-wise deletion

Question 6

What is MAR (missing at random)?

Answer 6

When missingness is conditionally independent of the unobserved estimate of our parameter(s) of interests
Conditionally dependent on values of variable we observed
E.g dog eats homework only if the student is female

Question 7

What is MNAR (missing not at random)?

Answer 7

Missingness is associated with the estimate of our parameter(s) of interest
Dependent on unobserved/uncollected values of the outcome of interest
E.g dog eats bad homework, missingness = directly related to the parameters of interest

Question 8

Will listwise deletion yield biased or unbiased estimates for
a) MCAR?
b) MAR?
c) MNAR

Answer 8

a) Unbiased
b) Possible to recover unbiased estimates if the variables missing values are conditioned on are present
But biased if only using complete cases (i.e from list wise deletion)
c) Cannot recover unbiased estimates (data needed to recover them is itself missing)

Question 9

What are multiple imputations?

Answer 9

Estimation of new value for the missing value over multiple datasets

Question 10

What is the outcome of multiple imputations?

Answer 10

Pooled estimates (includes sampling uncertainty (V hat) so it’s not called average estimates)

Question 11

What type of uncertainty/variation is calculated in each imputed dataset?

Answer 11

Sampling variation, overall uncertainty estimate (V hat)

Question 12

What 3 types of variation/uncertainty does the average of a set of imputed estimates (i.e. pooled estimates) have? These add up to total uncertainty

Answer 12

1. Sampling variation (V hat)
2. Between-imputed dataset variation / Missing data uncertainty
   (estimate of uncertainty from changes in imputed missing data itself across imputed datasets)
3. Uncertainty from not generating an infinite number of imputed datasets (B/M)

Question 13

What can you do to lower the level of uncertainty in multiple imputations?
Is it a good solution?

Answer 13

Increase m, the number of imputed datasets
No, get diminishing returns and slow analysis

Question 14

How many imputed datasets are recommended?

Answer 14

25-100

Question 15

MI steps in R (step 1)
What dataset do you start with?

Answer 15

Raw dataset with missing data

Question 16

MI steps in R (step 2)
What are missing values filled with?

Answer 16

Initial estimates e.g mean, random numbers from observed data range

Question 17

MI steps in R (step 3)
What model is built for each variable with missing data using other variables?

Answer 17

Prediction model, GLMs commonly used as these models but can use any

Question 18

MI steps in R (step 4)
What is the prediction model used for?

Answer 18

To predict missing data
These predictions do not capture uncertainty. We only want uncertainty in imputations, not data (predictions) used to build imputation model

Question 19

MI steps in R (step 5)
What steps are repeated for a complete dataset (reach convergence)?

Answer 19

step 2 (fill in missing values with initial estimates)
step 3 (build a prediction model based on step 2)

Question 20

Are iterations and multiple imputed datasets the same thing?

Answer 20

No. Iterations occur at each MI dataset.

Question 21

What is the problem with prediction models with GLMs when using small datasets (e.g 100 people)?

Answer 21

GLMs require a sample size larger than the number of predictors
In small datasets= possible to have more variables than people

Question 22

What is the problem with prediction models with GLMs that specifies interactions?

Answer 22

GLMs only include interactions between variables when specified by analysts, and often don’t know what that needs to be

Question 23

What does the margin plot (augmented scatter plot) show?

Answer 23

Variable X value when Variable Y is missing (conditional missingness)
Scatterplot of non missing data between 2 continuous variables with margins showing where missing data are

Question 24

What are 2 ways in which identifying patterns of missing data is useful?

Answer 24

1. Whether there’s enough overlap of relevant variables to predict missingness
2. Unexpected patterns that might indicate problems with data

Question 25

What are 2 tests to conduct to see if 1 variable is significant/systematically related to another variable’s missingness?

Answer 25

1. Welch 2 sample T-test
2. Pearson’s chi-square

Question 26

What does it mean when data reaches convergence?

Answer 26

No clear systematic change with additional iterations

Question 27

When using densityplot() or xyplot() to plot imputed vs. observed data, what colour are they shown in?

Answer 27

Imputed = red
Observed = blue

Question 28

If observed and imputed distributions are similar, what is this indicative of?

Answer 28

data being MCAR or that relevant variables are missing from model

Question 29

What do these outputs mean using the pool() function:
estimate, ubar, b, t, dfcom, df, riv, lambda, fmi

Answer 29

estimate: pooled, average regression coefficients
ubar: average sampling variance (V hat)
b: between imputation variance (missing data uncertainty)
t: total variance (ubar, b, simulation error)
dfcom: df for complete data analysis (if there had not been missing data)
df: estimated dfs accounting for missingness
riv: relative increase variance due to missing data
lambda: proportion of total variance due to missing data
fmi: fraction of missing information

Question 30

What do these outputs mean using this the “summary( pool (mi.reg), conf.int = TRUE)” function:
estimate, std.error, df, p-value, 2.5% and 97.5%

Answer 30

estimate: pooled average regression coefficients
std.error SE of estimates incorporating all 3 uncertainties (ubar, b, simuation error)
df: estimated df incorporating uncertainty due to missing data
p-value: probability of obtaining estimate in sample given true population value was 0
2.5% and 97.5% = lower and upper limit of 95% CI

Question 31

What does the function aggr() do

Answer 31

Creates plot of missingness by variable and patterns of missing data for an entire dataset

Question 32

What does mice() function do

Answer 32

For multiple imputation
Takes dataset, number of imputations + iterations