Multiple regression Flashcards by Sarah Ashmori

what are the assumptions of regression?

residuals normally distributed (mean of 0 and SD of sigma squared)
homoskedasticity (Variance of residual remains constant no matter the value of x
Residuals not correlated

How well did you know this?

Not at all

Perfectly

how exactly do we get the regression line? what method is used

Line is fit using the method of least squares.where the square of the residuals is minimised.

How well did you know this?

Not at all

Perfectly

what are we estimating with regression?

the intercept and slope of the population parameter

How well did you know this?

Not at all

Perfectly

why center a variable?

to allow for more meaningful interpretation. E.g, imagine centering age to 46, the mean age of a sample. Then the intercept tells us what the DV id for a 46 year old with all other predictors set to 0.

How well did you know this?

Not at all

Perfectly

if you standardise a variable e.g., age, how does this impact the interpretation of the regression line

what now does the slope reflect?

After standardising a variable, the slope is interereted as a single SD change, how this would affect the outcome.

How well did you know this?

Not at all

Perfectly

if i standardised both the predictor and outcome variable. what does the coeefficient for the X variable reflect?

pearsons correlation coefficient

How well did you know this?

Not at all

Perfectly

in simple regression what part of the equation captures the explained and unexplained variance?

systemic part : B0 + B1
Random part: residual (e)

the

How well did you know this?

Not at all

Perfectly

describe the form of a regression equation

Response = Systemic part + random part

How well did you know this?

Not at all

Perfectly

in simple regression how do we capture the residual variability?

sigma squared

How well did you know this?

Not at all

Perfectly

to estimate the residual variability using sigma squared - what assumption needs to be made

that the residuals are normally distributed

How well did you know this?

Not at all

Perfectly

What is explained variance in Y, unexplained variance in Y and the total variance

Explained variance = the variance in Y that is explained by X
Unexplained variance = residual
Total variance = sum of explained and unexplained variance.

How well did you know this?

Not at all

Perfectly

what is R-squared? What is the formula to calculate this?

The amount of variance in Y explained by X
Formula: explained variance / total variance = R-squared

How well did you know this?

Not at all

Perfectly

in simple regression what is R squared the same thing as?

The square of the Pearson’s correlation coefficient

How well did you know this?

Not at all

Perfectly

what two things affect the standard error

> sampel size

> amount of variability in X and amount of variance in Y unexplained by X (residual variance)

specifically, SE DECREASES with more variability in X, SE INCREASES with more residual variance.

How well did you know this?

Not at all

Perfectly

How use the SE to calculate the 95% confidence interval for the slope estimate (B1)

lets say SE is 0.001

CI = slope +/- (1.96 x 0.001)

How well did you know this?

Not at all

Perfectly

how do we use the SE to calculate the test statistic (aka the Z or t-ratio)

SE of sex is 0.025
coefficient is -0.156

slope / SE = test statistic

so Z ratio is -0.156 / 0.025 = -6.12

How well did you know this?

Not at all

Perfectly

we have 2 groups treatment and control. We want to test whether there is a difference int the variance of these groups.

I could use ANOVA or regression. Which is better?

Regression as it allows you to control for the effects of other predictors

How well did you know this?

Not at all

Perfectly

multiple regression with categorical predictor. measuring score of hunger in different countries

Germany, UK , france,

UK is reference coutnry.

HUNGER = BO + B1 + B2 + E

what exactly do the B1 and B2 slope reflect

B1 = the difference in means Germany vs UK
B2 = difference in means France vs UK

How well did you know this?

Not at all

Perfectly

how can we express the null hypothesis for the difference in hedonism scores for Germany vs UK. Then again for France vs UK?

H0: B1 = 0
HO: B2 = 0

How well did you know this?

Not at all

Perfectly

multiple regression with categorical predictor. measuring score of hedonism in different countries

Germany, UK , france,

hedonism = BO + B1 + B2 + e

what does the regression with dummy coded variables tell us

the mean hedonism score for n in
- uk
- germany
- france

multiple regression with more than one explanatory variable.

Y = b0 + b1x1 + b2x2 + e

what value do we expect each predictor to be at the level of the intercept?

multiple regression with more than one explanatory variable.

Y = b0 + b1x1 + b2x2 + e

interpret the coefficients B1 and B2

B1 the coefficient for variable X1, interpreted as a change in Y for a single unit change in X1 while controlling for x2.
Likewise X2 is the change in x2

multiple regression with more than one explanatory variable.

Y = b0 + b1x1 + b2x2 + e

what does the residual mean in this equation?

The variance of Y that isn’t accounted for by X1 nor X2

multiple regression with more than one explanatory variable.

Y = b0 + b1x1 + b2x2 + e

what is the null and alternative hypothesis for this equation?

H0: Bk = 0
H1: Bk != 0

How can we test for significance in multiple regression?

By examining confidence intervals for each parameter or, equivalently, by comparing Z-ratios to the normal distribution and calculating a p-value.

lets say we have a linear regression analysis with 1 categorical PV. Is this linear or multiple regression?

* Multiple as there are two variables included in the equation. Dummy coded.

what are the parameters in multiple regression?

intercept and slope

what is the standardized coefficient of the predictor X?

This is the lsope we expect when both X and Y are standardised before the analysis.

what statistic in simple regression is the standardised coefficient for the predictor X equivalent to?

Pearsons correlation coefficient

interpret what the standardised coefficient for predictor X1 in multiple regression with 2 predictors

The change in SD for 1 unit increase in X1 on the DV, while holding X2 constant.

what does 1 unit of a standardized variable correspond to?

1 SD

imagine a multiple regression model of hedonism on age and education, the standardised coefficient for AGE is -0.358. Interpret the relationship between age and hedonism.

1 SD change in age, predicts a .358 decrease in the SD of hedonism.

how do we know if we have a linear relationship between the x and y variable

plot them against eachother. If there is a curve then the relationship is non linear and we should fit a quadratic function

we have fit a quadratic curve to the relationship between age and hedonism. age is included as both a linear and squared term. linear age has a negative coefficient and squared age has a positive coefficient. what does the positive coefficient of the squared term, together with the negative coefficient of the linear term tell us about the relationship between hedonism and age?

that the negative relationship flattens out at older ages

R squared

* The proportion of variance in Y that is explained by all variables in the model * Also reflects the square of the correlation between the predicted and observed Y values in the model

R2 vs Adjusted R2

R2 will always increase even if irrelevant variables are added to the model. It is then common to use the adjusted R2. Thit is better because it takes into account just how many variables you are including in the model. It is a measure of the goodness of fit that’s penalises you the more variables you include. If you add variables, its value will only increase If their addition explains some variance in Y adjusted is better - More meaningful interpretation of the value of the new predictors. Value will only increase if they explain some variance in Y.

interpret R2 for both simple regression and multiple regression

In simple regression – the varance in Y that is accounted for by X. In multiple regression it is the variance in Y that is accounted for by all predictors in the model.

what does an R2 of 0.121 tell us?

That 12.1% of the variation in hedonism scores is due to variation in age and education. The correlation between the predicted and observed hedonism score is 0.348. Square root of this value is 0.121

What does R2 tell us about the predictability of the model?

It tells us the square of the correlation between the predicted values of Y (from the fitted model) and the observed values of Y

multicollinearity. Why is high correlation between predictos bad?

The coefficient estimates will be unstable and imprecise (large SE). Solution: either drop one or create a new variable that is a combination of the two.

when allowing an interaction term (age *sex) in multiple regression model. why does the multiplication of age and sex into a new variable help us see interaction effects

Interaction effect allows for the influence of age on hedonism score to differ for men vs women. If this is significant then we say there is an interaction effect.

multiple regression. predicting hedonism with age and gender. Q: how would the regression model differ for men vs women? Men coded as 0 and women as 1. Hed(i) = b0 + b1*(AGE)i +b2*(SEX)i + B3*(AGE*SEX)i + Ei

intercept and slope for men * B0 = intercept * B1* AGE = slope intercept and slope for women *B0 + B2 = Intercept, b2 being the difference in intercept for women from men *B1 + B3 = slope, b3 being the difference in slope for women from men

difference between a one sided and two sided test

one sided: testing that the test statistic is greater (or less than) than X.XX only. two sided: the test statistics is either + than 3.33 or less than -3.33

for a test investigating whether the influence of cohort on score depends on social class, what would the null hypothesis for no interaction existing be?

That the coefficients for all the interactions are 0 H0: B5= B6 = B7 = 0.

measuring the influence of coutnry (uk, germany, france) on hedonism score. how can we test whetehr the 3 countries all have the same slope?

include interaction term in model. Then compare this to the results of a simple model (nested within). if a significant difference between the two tells us at least 1 of the coutnries slope differs from the others

what test do we use to compare two nested models.

F test So when investigating whether 1 coefficient is equal to 0 , we can use the t statistic. When we want to investigate a group of coefficients, use a nested F test.

What is the f statistic?

The difference between two R2 values for 2 models, the interaction model and the restricted model. The change in R2 is compared to the F distribution.

how can we calculate the Wald statistic using the Fstatistic ?

Wald = no. of parameters constrained at 0 (interaction coefficients) X F e.g., for a model with 3 interaction terms and an f statistic of 46.56 3 × 46.56 = 139.68

What is the equivalent of comparing the wald statistic and chi squared distribution

Comparing the Wald statistic to a chi-squared distribution is equivalent to comparing the F statistic to an F-distribution.