06.a Linear Regression

Question 1

In linear regression what is the output variable called and what are the input variables called

Answer 1

Output variable = dependent variable

Input variable = independent variable

Question 2

What is linear regression

Answer 2

Linear regression is an analytical technique used to model the relationship between several input variables and a continuous outcome variable. A key assumption is that the relationship between an input variable and the outcome variable is linear. Although this assumption may appear restrictive, it is often possible to properly transform the input or outcome variables to achieve a linear relationship between the modified input and outcome variables.

Question 3

What is the theory behind linear regression

Answer 3

Regression analysis attempts to explain the influence that a set of variables has on the outcome of another variable of interest. Regression analysis is a useful explanatory tool that can identify the input variables that have the greatest statistical influence on the outcome.

Question 4

Name four common use cases for Linear Regression

Answer 4

Real estate
Demand forecasting
Medical correlation
Engineering

Question 5

What is the relationship expression for Linear Regression

Answer 5

Y = β0 + β1X1 + β2X2 +…+ βnXn + ε
Y = dependent variable (continuous outcome variable)
Xj = independent variables (input variables, j = 1, 2, …, n)
β0 = intercept (the value of Y when each Xj equals 0)
β1..n = coefficients
ε = error between data and the model
We know Y and Xj (historical dataset) and we have to find the regression coefficients (β0, β1..n)

Question 6

What does OLS mean and what does it do

Answer 6

Ordinary Least Squares. The goal is to find the line that best approximates the relationship between the outcome variable and the input variables. With OLS, the objective is to find the line through these points that minimises the sum of the squares of the difference between each point and the line in the vertical direction.

Question 7

Name three graph types which are used in Linear Regression

Answer 7

Scatter plot - visualise the linear relationship
Box Plot - to help spot outliers
Denisty Plot - to see the distribution of the predictor variable. Ideally, a close to normal distribution (a bell shaped curve), without being skewed to the left or right is preferred.

Question 8

What is the correlation coefficient in Linear Regression

Answer 8

Thecorrelation coefficient (r) quantifies both the strength and direction of the linear relationship between two measurement variables on ascatterplot. 1 = perfectly uphill, 0 = no correlation, -1 = perfectly downhill

Question 9

What do Anscombes Quartet show

Answer 9

The data sets in the Anscombe’s quartet are designed to have approximately the same linear regression line (as well as nearly identical means, standard deviations, and correlations) but are graphically very different. This illustrates the pitfalls of relying solely on a fitted model to understand the relationship between variables.

Question 10

Where is a good place to start in R when assessing if variables have correlation

Answer 10

Use the scatterplot to show a matrix of plots of all combinations of variables and look for correlations

Question 11

What is R squared

Answer 11

R squared is the coefficient of determination and is the proportion of the variance in the dependent variable that is predictable from the independent variable(s)

It ranges from 0 to 1 where 1 is an excellent fit and 0 is no fit.

It is the comparison of the variance compared to the model variance

Question 12

What is the syntax in R for linear regression

Answer 12

results = lm(y ~ x1 + x2, data = set1)

Question 13

What is a quick function in R to draw a straight line onto a scatter plot

Answer 13

abline

Question 14

What is Heteroscedasticity

Answer 14

It is the increasing spread of data away from the line of best fit for increasing data point values. The opposite is Homoscedasticity.

Question 15

What is Multicollinearity

Answer 15

Multicollinearity means there are relationships between variables. This can impact the interpretation of a linear regression model.

Question 16

What is a Q-Q plot and what is it used for

Answer 16

A Q–Q (quantile-quantile) plot is a probability plot, which is a graphical method for comparing two probability distributions by plotting their quantiles against each other. Good at showing a difference from a normal distribution.

Question 17

What is N-Fold Cross-Validation

Answer 17

N-Fold Cross Validation is a technique employed when you don’t have sufficient data for a standard supervised training and test approach. It involves taking splitting the data set into N subsets. Then N-1 data sets are used as the training data set with the remaining one used as the test. Repeat for every subset. Average the errors across the N-Folds.

Question 18

What are the residuals in Linear Regression

Answer 18

The residuals are the differences between the actual dependent variable data points and the modelled dependent variable data points.

Question 19

How do you evaluate the residuals in Linear Regression

Answer 19

Using the OLS method. Note that an R squared result of 1 means the residuals are all zero, which may suggest over-fitting.

Question 20

How do you calculate variance

Answer 20

Calculate the mean
Determine the difference from mean for each term
Square those differences
Sum the results
Divide by n-1