Chp3 Regression

Question 1

Linear Regression Theory

Answer 1

A statistical process for estimating the relationship among variables

Question 2

What two variables are in regression?

Answer 2

Response Variable (dependent variable y)
Predictor Variables (independent variables, X)

Question 3

What is regression used for?

Answer 3

Predicting and forecasting

Question 4

What does linear regression try to do?

Answer 4

Use past data to predict future outcomes

Question 5

What are model coefficients/parameters/weights?

Answer 5

multiply them against the input values to generate your response variable

Question 6

What is error called between observed response Yi and predicted response Yhat

Answer 6

Residual

Question 7

Residual Formula

Answer 7

Yi - Yhat
observed response - predicted response

Question 8

Residual Sum of Squares

Answer 8

RSS, sum of each residual squared

Question 9

Residual Formula mathematical simple

Answer 9

Ei = (B0 + B1Xi) - (Bhat0 + Bhat1X1)

Question 10

Which is the best regression line?

Answer 10

The one that minimizes the sum of squared residuals

Question 11

Multiple R-squared

Answer 11

Will always increase as you add more predictors because increasing variance and every predictor is increasing multiple R-squared, but not every predictor is a good predictor

Question 12

Adjusted R-squared

Answer 12

Captures how many of those predictors you have added are actually good predictors as you add those predictors. Mult r sq and adj r sq values go up, but there will be a time where the adj r sq will plateau and drop, stop adding variables at that point

Question 13

Adjusted R-squared shows

Answer 13

When adding more predictors makes it worse

Question 14

F statistic

Answer 14

Captures how good model is, bigger the better

Question 15

Degrees of freedom

Answer 15

How much wiggle room you have in your data set

Question 16

In hypothesis testing, what must be true to support the Null hypothesis H0

Answer 16

Pvalue > alpha

Question 17

In hypothesis testing, what must be true to not support the Null hypothesis

Answer 17

Pvalue <= alpha

Question 18

What is alpha in hypothesis testing

Answer 18

The probability of Rejecting the null hypothesis given that the null hypothesis is true

Question 19

What is pvalue in hypothesis testing

Answer 19

The probability of getting a result as extreme as you have given that the null hypothesis is true

Question 20

What are the only two outcomes from hypothesis testing?

Answer 20

• Reject H0 in favor of H1
• Do not reject H0

Question 21

In hypothesis testing, we never accept //

Answer 21

H1

Question 22

If we are looking if a drug has an effect, what is null and alternate hypothesis?

Answer 22

null - drug has no effect
alt = drug has some effect

Question 23

What are the four questions to evaluate the fit of a regression model?

Answer 23

Is at least one of the predictors useful in predicting the response?
Do all the predictors help explain the response, or is only a subset of the predictors useful?
How well does the model fit the data?
Given a set of predictor values, what response value should we predict and how accurate is our prediction?

Question 24

What is the hypothesis test to determine if at least one predictor is useful in predicting the response?

Answer 24

H0: all betas are 0
H1: at least one beta is nonzero

Question 25

What does it mean if the F statistic is close to 1?

Answer 25

There is no relationship between response and predictors, H0 is true

Question 26

What is f statistic if H1 is true

Answer 26

F-statistic > 1

Question 27

If at least one predictor is useful, what does that mean about the p value associated with the f statistic?

Answer 27

it is very small

Question 28

What is hypothesis testing for determining if all the predictors explain the response, or if only a subset of predictors are useful?

Answer 28

For each predictor H0: Bi = 0 (there is no relationship between predictor and response) H1: Bi != 0 (There is some relationship between predictor and response)

Question 29

What p value makes a predictor useful? not useful?

Answer 29

Very low P value a non low p value means we cannot reject the null hypothesis

Question 30

T value

Answer 30

Tells you how many std your beta value is from 0

Question 31

T value the farther

Answer 31

the better

Question 32

In example, what is significance that radio and newspaper have correlation in correlation matrix?

Answer 32

If we spend more on radio we are also spending more on newspaper, which should not be done because newspaper is not very correlated with sales

Question 33

Given p predictors how many models could we create

Answer 33

2^p

Question 34

two strategies for feature selection

Answer 34

Forward Selection Backward Selection

Question 35

Forward Selection

Answer 35

Start with null model (intercept) and add predictors

Question 36

Backward Selection

Answer 36

Start with all predictors and remove variables with largest p-values

Question 37

Which statistics 3 describe how well the model fits the data?

Answer 37

Residual Standard Error RSE R^2 RMSE

Question 38

what is r^2

Answer 38

Correlation between y and yhat

Question 39

What do we want r squared to be?

Answer 39

1, we want what we are predicting to be very correlated to the training data

Question 40

Residual Analysis

Answer 40

Sees how well the model fits the data

Question 41

Residuals should be 4

Answer 41

homosceadastic Centered on 0 through a range of fitted values normally distributed uncorrelated with each other

Question 42

Homosceadastic

Answer 42

Variance in residual must not vary too much

Question 43

RSE must be

Answer 43

minimized

Question 44

If residual plot is nonlinear, what type of regression would be best?

Answer 44

quadratic or higher power regression

Question 45

Significance level (alpha)

Answer 45

Probability of making the wrong decision, given H0 is true

Question 46

Confidence Interval

Answer 46

The range of results that would be expected to contain the population parameter of interest

Question 47

Confidence Level

Answer 47

Probability that if an experiment was repeated multiple times, results will be the same

Question 48

Confidence level formula

Answer 48

1 - alpha

Question 49

What does 95% confidence interval mean?

Answer 49

If I ran the experiment 20 times, then my true value (actual mean maybe) will be present in this interval 19 times.

Question 50

Higher confidence means

Answer 50

Higher margin for error, wider confidence intervals

Question 51

prediction interval is _ than confidence interval

Answer 51

wider

Question 52

Difference between prediction interval and confidence interval

Answer 52

Prediction interval is trying to predict something, so we have to take into account the variation in the data and uncertainty in knowing the true population parameter Confidence interval is basically drawn from the data you have

Question 53

Multi-collinearity

Answer 53

When one predictor is a linear combination of another predictor

Question 54

In the presence of multi-collinearity

Answer 54

Regression model will not work. Cannot trust the weights because when updating one weight you hold the others constant, which cant be true in multi-collinearity.