L2 Multiple Regression

Question 1

What is multivariate analysis?

Answer 1

Analysing the impact of multiple variables for predicting changes in Y.

Question 2

What are the two types of multivariate analysis?

Answer 2

Dependent and independent

Question 3

What are two analysis of dependence?

Answer 3

multiple regression and discrimination

Question 4

What are three analysis of independence?

Answer 4

cluster analysis, principal component analysis, factor analysis

Question 5

What is the purpose of multiple regression?

Answer 5

It is designed to isolate the effects of each x-variable (predictor) upon the y-variable

Question 6

What is a coefficient?

Answer 6

A numerical description of an x-variable’s relationship upon the y-variable

Question 7

What is a regressor/predictor?

Answer 7

an independent variable

Question 8

What is the model?

Answer 8

term used to describe the collection of regressors involved in predicting y

Question 9

Given an example of a multiple regression case and 5 potential regressors?

Answer 9

Plant growth rate (y) is dependent upon 1) temperature, 2) radiation, 3) carbon dioxide, 4) nutrient supply, 5) water

Question 10

How and WHY does the line of best fit changes for multiple regression?

Answer 10

Because there are more x-variables, their relationship is now collectively non-linear so the line becomes a plane of best fit

Question 11

How does the r^2 statistics differ in multiple regression?

Answer 11

it becomes r^2v(adj) [adjusted r squared value].

Question 12

What is the adjusted r squared metric and why does it differ from its linear counterpart?

Answer 12

The linear r squared considered how much of y was explained by a singular predictor. Whereas the adjusted version measures how well the whole model explains variance in Y.

Question 13

What happens to the adjusted r squared value if you add more regressors to the model and why?

Answer 13

The adjusted r squared can either increase or decrease. This is because the other regressors may either contribute an increase to the understanding/explanation of y, or they may not be relevant and so cloud out/decrease the model’s overall explanation of Y.

Question 14

What is crucial before we compare adjusted r squared values?

Answer 14

Standardizaiton

Question 15

Why do we need to standardize in order to compare adjusted r squared values?

Answer 15

because of the different data, scales and models between different samples that make it inappropriate to compare

Question 16

What does scale dependency mean and why is it important?

Answer 16

In multiple regression because there are different regressors inputted in to the model, of which they have different units of measurement, it means that when you add another regressor, the initial ones change i.e. their scale is dependent upon the others. Importantly however the way they change is not necessarily correct because of the different units of measurement between the different variables

Question 17

What is an example of scale dependency being important ?

Answer 17

Plant growth rate - water is measured in millilitres, carbon dioxide is measured in ppm usually. This means that they respond in incorrect ways

Question 18

What technique allows us to overcome the scale dependency?

Answer 18

standardization

Question 19

What happens to the name of the coefficients as they are standardised?

Answer 19

they transform from partial regression coefficients in to beta regression coefficients

Question 20

To overcome the different units of the different regressors what happens to the units?

Answer 20

They become z units

Question 21

What is the t value?

Answer 21

the significance of the coefficient in explaining the variance

Question 22

What is multicollinearity?

Answer 22

When the regressors in a model correlate with each other

Question 23

Why is multicollinearity a problem for multiple regression?

Answer 23

Because multiple regression seeks to determine the specific effect of each regressor separately upon Y.

Question 24

If multicollinearity is present, then how would the finding of one regressor impacting Y in a certain way be hampered?

Answer 24

because a regressor correlates with another regressor then the impact of the initial regressor upon Y would have to be shared between both regressors, and there would be no way of determining which one is a more important regressor. Furthermore, they may both be controlled by an underlying factor

Question 25

How do we look for multicollinearity?

Answer 25

using the variance inflation factor

Question 26

What are the 3 ways we can address multicollinearity?

Answer 26

1. Remove one or more of the correlated regressors and then re-run the test 2. Where there is a logical relationship between the two variables we can create an interaction term 3. If variables are related because of underlying factors then they can be reduced using factor analysis

Question 27

What does the f-test enable?

Answer 27

the determination of how taking a regressor out of the model impacts the Y value

Question 28

How does the f-test result work?

Answer 28

Look to see how much the f-test value changes between different models with different regressors included or excluded.

Question 29

How do you interpret the f-test result?

Answer 29

If F changes a lot then it means the inclusion or exclusion of that regressor has a big impact upon Y. Furthermore, the f-value will also have a p value which indicates the statistical significance of that result.

Question 30

What are 3 things to remember when we want to undertake a meaningful multiple regression analysis?

Answer 30

1. Variable selection - dont chuck in a load of variables and see what happens, instead select them and their order based on theory and previous findings. 2. Do not rely on a single metric to assess, instead use multiple as they all have a specific and useful purpose. 3. Parsimony - do not overcomplicate the model, seek coherence and simplicity.

Question 31

What are the 3 methods for building the structure of the model's regressors?

Answer 31

1. Hierarchical: ordering the regressors individually, based on theoretical considerations with known predictors entered in first and in order of importance 2. Step-wise: order based on mathematical reasoning, which means that is is usually carried out by computers based on a correlation match rate (predictors that explain most are ordered first) 3. Forced-entry/simultaneous: all regressors (known or unknown) are put in to the model in a simultaneous manner but all regressors have been considered before

Question 32

What is important to remember for the validation of a model?

Answer 32

Cross-validation

Question 33

What is cross-validation?

Answer 33

Applying the model with the same predictors for a different sample or population

Question 34

What two things are important to consider about your model once you have produced it?

Answer 34

Does it fit the observed data well (or is it overly affected by a few regressors), and can the model be generalised to other samples

Question 35

What is the standardization formula?

Answer 35

beta coefficient (standardized multiple regression coefficient) = partial regression coefficient x (standard deviation for regressor divided by standard deviation of variable y)

Question 36

How do we determine multicollinearity in SPSS?

Answer 36

Look at the tolerance and VIF values. Ideally we want tolerance to be >0.2 and the VIF to be <5.

Question 37

What do the tolerance and VIF metrics mean?

Answer 37

``` tolerance = e.g. if = 0.3 then this means 30% of the variance for that given predictor is not accounted for by the other predictors. We therefore want it to be higher than 0.2 so that we know that the predictor is not useless because other predictors explain it themselves. VIF = if >5 this indicates the presence of multicollinearity ```

Question 38

How doe we determine heteroscedasticity in SPSS?

Answer 38

We look at the scatterplot, histogram and the p-p plot.

Question 39

What is heteroscedasticity? Why does it matter?

Answer 39

When our residuals are not constant across the x-axis. If this occurs then we cannot be as confident in our regression analysis

Question 40

How do we interpret the scatterplot, histogram and p-p plot in SPSS to determine heteroscedasticity?

Answer 40

Scatterplot = we want to see a random and wide spread for homoscedascity Histogram = we want to see a normal distribution (bell-shaped curve) P-P plot = we want to see a fairly even alignment of points to the line

Question 41

What is autocorrelation? Why is it a problem?

Answer 41

This is when each of our correlations is not independent of the others around it. Regression assumes that they are independent

Question 42

How do we determine auto correlation in SPSS?

Answer 42

Look at the Durbin Watson statistic?

Question 43

What is the range of Durbin-Watson statistics where we would state there was no autocorrelation?

Answer 43

1.566 - 2.434

Question 44

What is the range of Durbin-Watson statistics where we would state there is significant positive or negative autocorrelation?

Answer 44

``` Positive = 0.1.475 Negative = 2.525-4.0 ```

Question 45

What are the ranges of Durbin-Watson statistics where we would state there is indeterminate autocorrelation?

Answer 45

1.475-1.566 & 2.434-2.525

Question 46

How does stepwise regression work in SPSS?

Answer 46

SPSS uses the correlation matrix to find the independent variable that has the largest significant Pearson's correlation with Y, followed by the next and then orders them in that order.