Linear Regression

Question 1

What is linear regression?

Answer 1

A tool for describing relationships between multiple interval scale variables

Question 2

In linear regression, the outcome and predictor are?

Answer 2

The outcomes and predictor are both numeric

We can have multiple predictors

Question 3

What is the fundamental idea for linear regression?

Answer 3

Fit the best regression line to the data, then try to understand that line

Question 4

What is the formula for a linear regression line?

Answer 4

Question 5

Changing the intercept of the regression line does what to the line?

Answer 5

Raises or lowers the regression line.

Question 6

Changing the slope of the regression line does what to the regression line?

Answer 6

Alters the steepness of the regression line

Question 7

How do we know what the best regression line is?

Answer 7

The principle of ‘least squares’

Question 8

What is the principle of least squares?

Answer 8

The best regression line for data (X,Y) is the one that minimises the sum squared deviation between the predictions and the actual values

This is referred to as the residual sum of squares

Question 9

How do you estimate a simple linear regression model in R?

Answer 9

Like ANOVA, it is done in stages

1. lm() estimates the values of b0, b1 etc
2. summary() runs some hypothesis tests

The lm() function

This is the main “workhorse” function

It creates an “lm” object (i.e. variable), which contains lots of quantities of interest relating to regressions

Question 10

What is a regression with only a single predictor called?

Answer 10

Simple linear regression

Mostly the same thing as Pearson correlations

Question 11

In real life, we usually expect that multiple variables could predict our outcome, what is this called?

Answer 11

Multiple linear regression

Question 12

The slope terms and intercept are called _____

Answer 12

Regression coefficients

Question 13

What is the test statistic in a multiple linear regression?

Answer 13

F statistic

Question 14

What is the sampling distribution of T if the null is true in multiple linear regression?

Answer 14

F distribution

Exactly analogous of ANOVA

Question 15

The null hypothesis in ANOVA’s predict?

Answer 15

No relationship between the predictors and outcome

(all slope parameters are zero)

Question 16

The alternative hypothesis in ANOVA’s predict?

Answer 16

The relationship between the predictors and outcome matches the regression model

Question 17

What are the two sources of variance in a regression model?

(assume a model with K predictor variables)

Answer 17

SSmod and SSres

Question 18

How do you test the null hypothesis that a specific predictor variable has no relationship to the outcome?

Answer 18

Use a t-test

Question 19

What is the effect size measure for multiple linear regression

Answer 19

R²

Question 20

How do you interpret the R² effect size measure?

Answer 20

Question 21

Doing a multiple linear regression in R?

Answer 21

Question 22

What does the stat block for the regression model look like?

Answer 22

Question 23

How do you do regression analysis when the predictors are on different scales?

Answer 23

Using standardised regression coefficients

• Convert all the variables to standard scores before running a regression so all are on the same scale​*
• Usually denoted as β (beta)​*

Question 24

What are some of the dangers of using regression analysis?

Answer 24

1) Outlier

(Model prediction might be way off with outliers)

2) High leverage point

(An observation that has different values on the predictors than the other ones, residual might still be small though)

3) High influence point

(An outlier with high leverage, These are dangerous)

Question 25

Does regression correlation = causation?

Answer 25

No

Variables are not controlled and no intervention has been done, so causation cannot be inferred