Regression diagnostics / Logistic regression

Question 1

Why do we need to consider the assumptions for a linear regression?

Answer 1

So we can rely on those statistics: coefficients, SE, etc.

Question 2

Which regression assumptions are there?

Answer 2

differs a bit, but roughly:

Question 3

What happens if you violate regression assumptions?

Answer 3

1) Coeffcients become unreliable –> biased
2) SE become unreliable –> any hypothesis becomes unreliable (incluing p-value/t-stat, etc.)

Question 4

Linearity

Answer 4

Assumption: the average outcome is linearly related to each term in the model when holding all others fixed –> & technically, the “linear” in “linear regression” refers to the outcome being linear in the parameters, the β’s

Problem: Biased coefficient (true form is curvilinear)

Diagnostic: Component-plus-residual plot (A significant difference between the residual line and the component line indicates that the predictor does not have a linear relationship with the dependent variable)

Solution: Polynomial, Spline, Collapse into categories

Question 5

Homoskedastic / Normally distributed Residuals

Answer 5

Assumption: constant & normal variance of residuals

Problem: Standard errors usually not correct: underestimated, also, influential observations may be present can also effect coefficients

Diagnostic: Heteroskedastic –> Plot residuals, Normallity –> HIstograms, Qnorm, Studentized residuals plot

Solution: Log-transformation / Power-transformation, Robust SE‘s, Correct coding errors

Question 6

No multicollinearity

Answer 6

Assumption: Predictors should be independent of each other, very low correlated (not present in SLR but in MLR)

Problem: “holding constant” not possible with correlated variables –> 1) interpretation becomes impossible, also model will not know which varible made difference 2) Loss of precision (inflated standard errors)

Diagnostic: Look at correlations, Variance inflation factor (assesses each variable, what’s the difference in variance if we include/exclude it –> the higher the VIF, the more information is already contained = high multi-collinearity)

Solutions: Get crafty (similar but not collinear variable), Construct an index, Get more data, Mean-center interaction variables

Question 7

Which diagnostic is important to consider apart from regression assumptions?

Answer 7

Influential observations

• pull the regression fit towards themselves –> results (predictions, parameter estimates, CIs, p-values) can be quite different with and without these cases included in the analysis
• do not necessarily violate any regression assumptions, they can cast doubt on the conclusions drawn from your sample. If a regression model is being used to inform real-life decisions, one would hope those decisions are not overly influenced by just one or a few observations…

Question 8

What is more important: Coefficient vs SE?

Answer 8

First, estimation then SE, correct SE no use if estamtion is biased ..

Question 9

Answer 9

Question 10

Answer 10

Question 11

Answer 11

Question 12

Which one is problematic?

Answer 12

B - unusual value and large residual ergo leverage which pulls regression line down, deleting it would change regression line drastically

Question 13

Are these two influential observations?

Answer 13

NO
1) large sample
2) no unusual x-value

Question 14

When to delete an outlier?

Answer 14

Cook’s D of 1 approx.

Question 15

Answer 15

not a problem of bias per se but a lack of data, very little variance + hard to separate makes it hard to be precise

Question 16

Answer 16

a) hard decision to make

Question 17

Potential problems, diagnostics, potential solutions: Influential
observations

Answer 17

Question 18

Which is the link function?

OLS vs Logistic Regression

Answer 18

Linear = Identity link (just the mean –> model of the mean)
Logistic = Logit link (log odds of mean –> model of the log odds /also: logit)

Question 19

Which distribution?

OLS vs Logistic Regression

Answer 19

Linear = Gauss (continuos)
Logistic = Binomial (discrete)

Question 20

What are the problems with an OLS model when it comes to binary outcomes

Answer 20

1) binary outcome –> we want to model probability & linear model can give neg values but neg probabilities no meaning
2) unrealistic assumptions about constant effects
3) normal residual assumption is violated

Question 21

Interpret OLS model

Answer 21

1 unit increase in trust scale, decreases the mean of AFD vote by 0.05 percentage points

Question 22

What does the logistic regression model?

Answer 22

The logit

Question 23

How does ML work?

Answer 23

iterates to find best parameters

Question 24

How to interpret the Log Odds?

Answer 24

very hard, not intuitive ..

Question 25

How to interpret the Odds Ratio?

Answer 25

1 --> no difference

Question 26

Limitation of Odds

Answer 26

We can only say what the odds and how odds increase BUT NOT how likely something actuallly is (probabilites)

Question 27

How to get back to probabilities?

Answer 27

e to the power of function Example: plug in 0 for men --> constant = 10% plug in 1 for women --> constant - coefficient = 5%

Question 28

What is the average marginal effect?

Answer 28

Averages all the marginal effects of the dimensions of the explanatory variable-

Question 29

How do the average marginal effect and linear regression relate to each other?

Answer 29

very similar, lin reg approx. AME

Question 30

How to model this relationship with a linear model?

Answer 30

Polynomial of Trust in Parliament would pick up the slowing of the decline

Question 31

How to make this log reg table more interpretable?

Answer 31

Transform to probability scale (marginy, dydx) --> Average marginal effects (makes it more interpretable but also takes out the s-shape meaning it works well in the middle but less at the edges)

Question 32

Polynomials vs log regression

Answer 32

decline in non lin models

Question 33

What is the problem with interactions in logistic regressions?

Answer 33

log regression already uses interaction even if it is not explicit

Question 34

What is the problem with mediation in logistic regressions?

Answer 34

if you put in more, logistic mediation will be different, no cross-country comparison possible --> mediation generally underestimated

Question 35

What is the solution for logistic mediation?

Answer 35

KHB method (can also be used for linear models) --> Diff gives you mediation effect

Question 36

Answer 36

Important to consider the absolute as well as the relative ratio

Question 37

Logistic vs linear models | Binary variables

Answer 37

Question 38

Why would we prefer logistic regression sometimes?

Answer 38

Captures non-linear relationship really well, with OLS we need to model them ourselves