Regression

Question 1

What can we sa ythat r4egression is all about?

Answer 1

Trying to explain movement in one variable as a result of movement in other variables

Question 2

what is the difference between the explanatory variables X’s, and the explained variable Y?

Answer 2

Y is assumed to be stochastic, following a specific probability distribution.

The explanatory variables are assumed ot be fixed in repeated samples, i.e. non stochastic.

Question 3

why do we assume X to be fixed in repeated samples?

Answer 3

Remove randomness from this part. All randomness is allocated to the stochastic Y variable.

Question 4

Given a one-variable case where some financial theory suggest that increases in some X variable will lead to changes in Y, what is the first sensible thing to do?

Answer 4

Plot scatter to see if the pattern is linear

Question 5

what do we need to know about the equaiton

y = a x + b

Answer 5

It is an exact funciton. It is not realistic to see real appliations where the relationship is that exact.

Question 6

how do we include error term to the exact line y=ax+b?

Answer 6

We transform it to per-sample point:

y_t = a x_t + b + u_t

Or

y_t = a + b x_t + u_t

is more conventional.

u_t includes the difference between the exact line and hte specific data point.

Question 7

elaborate on reasons to include random disturbance term

Answer 7

1) The number of actual determinants (number of explanatory variables) is usually too large to be quantified perfectly, typically due to uonbservability etc

2) Some errors cannot be modeled

Question 8

how do we determine the correct line (linear regression model)?

Answer 8

Minimizing the sum of vertical distances between each point and the line.

This is correct because of the assumption of non stochastic explanatory variables.

Question 9

most cmmon method to generate the model line?

Answer 9

OLS

Question 10

Two other methods than OLS that can be used to find a model for linear regression

Answer 10

Method of moments

Maximum likelihood

Question 11

how do we find residual?

Answer 11

Difference between actual value y_t and predicted value for y_t

Question 12

what is the role of the residual in linear regression?

Answer 12

In OLS, we minimize the sum of these residuals (squared).

Question 13

broadly speaking, how do we find the functions for hte OLS parameters in the single-variable case?

Answer 13

Build loss function, differentiate the loss function with regards to the params, equate to 0, solve for params. This works if the loss function is convex. If convex, doing this will minimize the loss function, and provide us a set of parameters that give the smallest residual sum of squares.

L = ∑(u_t)^2 = ∑(y_t - (pred(y_t)))^2 = ∑(y_t - alpha - beta x_t)^2

Question 14

With OLS, what can we say about the MEAN of the datapoints?

Answer 14

The predicted OLS line will go through the mean points

Question 15

what do we need to understand about the intercept term?

Answer 15

It is given on the basis of how our data set look like. Typically, our data points will not necessarily cover the entire possible domain, as this is infeasible. Therefore, typically, we will have edges in data set, and if we try to go beyond these edges on either side, the results are essentially unknown since the model has not been fitted on these regions.

As a result, it is useful to also include some numbers on the “valid range” of our model.

Question 16

What is the difference between SRF and PRF?

Answer 16

Sample regression function does not include the error term, but the populaiton regression function does.

The SRF is the estimated line, while PRF is the “true” function.

Question 17

Is this linear?

Answer 17

it is linear in parameters, but not linear in variables. it can be converted to linear form, and use OLS to fit line.

Question 18

If theory suggests that Y is inversely related to X, can we use OLS?

Answer 18

Yes. We define new variable z = 1 / x, and use this to fit. Then we just need to transform observed values of X into Z before using the model to predict.

Question 19

elaborate on why this is

Answer 19

if we need to estiamte gamma, and the estimated value for gamma change either through time or through changes in the explanatory variables, then the linear model will not be accurate. However, if we estiamte gamma in a way that makes it behave as a constant, then we can use it with OLS.

Question 20

do we need to be specific about how the residuals are computed?

Answer 20

Yes, becasue Y_t depend on the residual

Question 21

The set of assumptions related to the classical linear regression model, do they apply to the predicted model or the unobserved values?

Answer 21

Unobserved. No assumptions are made on the values of the residuals that are found from the model that is predicted.

Question 22

elaborate on the CLRM assumptions

Answer 22

1) E(u_t) = 0

2) var(u_t) = sigma^2 < infinity

3) cov(u_i, u_j) = 0

4) cov(x_i, u_i) = 0

5) u_t is n(0, sigma^2), iid.

Question 23

elaborate on the assumption of normality

Answer 23

Necessary for hypothesis testing

Question 24

what can we say about the properties of the OLS parameters if the assumptions hold?

Answer 24

if 1-4 hold, we have BLUE.

Best Linear Unbiased Estimator

Best: the parameters have minimum variance

Linear: the parameters are given by fnctions that are linear

Unbiased: On average, the estimators will be equal to the true population parameters.

Estimator: estiamtoes of the true population parameters.

Question 25

Are the OLS estimators consistent?

Answer 25

yes

Question 26

what is consistnecy?

Answer 26

Probability of the difference between true paraemter value and estimated parameter value being larger than some constant will be 0 as the number of data points approach infinity. thus, consistency is about converging to true population values. Unbiasedness is just about average. Consistency say something about convergence. Consistency is an asymptotic property.

Question 27

elaborate on the B is BLUE

Answer 27

Best. refers to the fact that the estimator is the most efficient estimator.

Question 28

elaborate on the role of the sample in linear regression

Answer 28

Every lin reg is subject to the specific sample that is used. If sample change, results change. Therefore, it is useful to obtain an understanding of how good our measures are. meaning, how reliable are our parameters. The paremeters are subject to sampling variability. Therefore, we want ot quantify the sampling variability. We use standard error of the parameters to obtain this information. Standard error is the standard deviation of the estimator. It tells us how much we can expect the estimator to vary between different samples. It is found by computnig the variance of the OLS parameters (Var(alpha)) etc. Small are desirable.

Question 29

what is crucial to understand regarding stnadard error

Answer 29

it tells us nothing about how good the estimates are. It only tells us how much we can expect the estimated values to change from different samples. It is a measure of accuracy that does not relate to the model accuracy itself.

Question 30

what is the "standard error of the regression"?

Answer 30

Sample standard deviation of the regression model itself. It tells us how close the predictions will be to the actual data. However, again, it doesnt necessarily say that the accuracy of the estimators are perfect or not.

Question 31

what can we say about standard error estomates?

Answer 31

1) drop with larger sample size 2) Increase with greater variance in residuals, because SE(a) and SE(b) depends on S^2, which is the variance of hte regression

Question 32

when hypothesis testing on the OLS parameters, what distribution do we use?

Answer 32

We'd like to use normal, or standard normal. however, since the standard errors of the parameters are not known, as they cannot be observed, we need to use estimators. This requires us to use student t distribution instead.

Question 33

elaborate on the constant term

Answer 33

When representing data, we treat it as a variable as well. However, this variable will just have 1s as the values.

Question 34

what is k

Answer 34

k is the number of parameters that needs to be estimated. is therefore equal to the number of variables including the constant

Question 35

elaborate on the F-test framework

Answer 35

allows for multiple hypothesis at once. Two regressions are required: 1) Restricted 2) Unrestricted In regression, we get the statistic: F = (RRSS - URSS)/URSS x (T-k)/m The entire idea is that if the difference between the residual sum of squares is not large for the regression that has no restrictions and the regression that has restrictions, then the restrictions are backed by the data. If not, they are not backed by the data. This allows us to place constraints on the parameters, can set them to be anything. For instance, can set some of them equal to 0.

Question 36

how do we add a constraint to a regression?

Answer 36

Say we have b_1 + b_2 = 1 Then we can use "b_1 = 1 - b_2" and add this in place for b_1 in the regression. Then we gather all the b_i's on one side. This can mean that we need to move a variable (independent) to the other side, like this: y-x_4 = ... we create new variables, P=y-x_4 etc

Question 37

elaborate on dummy variables

Answer 37

Takes on either 0 or 1 Used to model qualitative variables into quantitative

Question 38

what is an intercept dummy?

Answer 38

dummy variable that works by changing the intercept, by shifting the regression up or down.

Question 39

elaborate on dummy trap

Answer 39

If we use dummies to encode categorical variable that has more than 2 options, we enter a trap if the sum of the variables are always 1. if this is the case, we get fucked up results. we need to have the possibility of the sum being equal to 0.

Question 40

elaborate on goodness of fit

Answer 40

We want a way to say sometihng about "how well the model is able to explain deviations in the explained variable about its mean". Consider if we simply take the mean of the data set, and call it a regression model. Then we measure the residual sum of squares. Then we fit the actual model, and do the same. We can think about the added precision of the model as explaining variance about the mean. Note that this is not about how well the model generalize. It is about how well the model is able to explain the variation in the sample. The most common one is R^2, and it is given by: R^2 = ESS/TSS, or equivalently TSS-RSS/TSS = TSS/TSS - RSS/TSS = 1 - RSS/TSS

Question 41

what is actually ESS

Answer 41

Explained sum of squares. It is a part of TSS. TSS = ∑(pred(y_t) - mean(y))^2 + ∑(u_t)^2 ESS is therefore a value that tells us how much deviation there is between a sample point and the mean sample point.

Question 42

R^2 = 0 indicates that...?

Answer 42

The model has not explained any of the variance about its mean. ESS = 0.

Question 43

prblems with R^2?

Answer 43

if the best fitting line is the mean, then we get R^2 = 0, even though the line may be very good. It is not sensible to compare R^2 values of models that have even just slightly different dependent variables. R^2 never falls when adding more independent variables.

Question 44

can we improve R^2?

Answer 44

Adjusted R^2 to account for penality of more regressors.

Question 45

elaborate on hedonic pricing models

Answer 45

The core idea is that one can represent total value by the sum of its individual parts. this gives rise to the possibility of modeling a price by simply adding properties. Typically used in real estate properties.