Chapter 3 - Intro to regressiobn Flashcards
(26 cards)
What is regression
Regression is a relationship between one variable and multiple others.
More specifically, regression is a way to describe movement in one variable as a result of movement in others.
The variable of interest is denoted “y”, and the variables we use to model y is x_i.
Name some other common terms for the “y” and “xs”
y:
- Dependent variable
- Regressand
- Effect variable
- Explained variable
X:
- Independent variables
- Regressors
- Causal variables
- Explanatory variables
what is measured by correlation
The degree of linear association between variables.
There is nothing about causality here.
A high level of correlation entail that there is a linear relationship between the two variables.
are there cases for regression with only one independent variable?
CAPM, because market risk is the only variable.
Stock price and dividends’ long term relationship
Constructing optimal hedge ratio
What is the first intuitive way to explore a 1-indpendent variable regression case?
Scatter plot 2D
what is a problem with the linear regression function:
y = a + bx
It is a best fit, not a perfect fit. There is always uncertainty involved in it.
Therefore, it it usually better to include a “random disturbrance” term in it:
y = a + bx + u
We can add observation number to be more precise:
y = a + bx_t + u_t
elaborate on why we add the random disturbance term to the regression model
It will realistically always be some factors that have causal relationships with the dependent variable that we omit from the model. It is just simply not possible to find everything. Therefore, we cannot expect certainty to 100% in our model.
We can also have measurement error in y.
Some things cannot be modeled.
elaborate on the “random disturbance term”
we have a bunch of samples (for both Y and the X-variables). Using these samples, we make the best fitting regression model. Now we have a hyperplane given by the equation y=a+bx, alternatively y=a+[b1,b2…][x1,x2…], that represent our model. Using this model, we can make new pass over the sample points, and for each point we can find the difference between the actual value and the model prediction. This gives us the residuals for each sample. With this information, we can for instance compute the mean squared error to get some sort of idea of the error we’re dealing with on average. Or we could use root mean squared error to get the same unit as the dependent variable use.
shortly, what do we do to tfit the regression parameters
Make them so that the sum of vertical distances for each point is minimized
name 4 (3+1) methods of finding the parameters to the regression model
OLS
Moments
Maximum likelihood
Gradient descent
give the intuition behind OLS
THe idea is that we want to minimize the sum of squared residuals.
elaborate on PRF and DGP
PRF: Population regression function
DGP: Data generation process.
The idea is that the PRF is the same regression line/hyperplane as the one we find using OLS, BUT with the “true” population parameters. Recall that we are always using the estimated values based on the samples we have drawn.
Elaborate on why we do not have random disturbance term in the PRF
We actually do. And the reason for this is the same as for why we did with the regression model built by the samples.
It is simply infeasible to find all causalities among variables. Some things are not possible because we dont know about the relations, and some things are not possible because we cannot model it simply. In either way, we add the random disturbance term.
what is SRF
Sample Regression function.
Basically our regression model that is built using our samples.
ultimately, we use SRF to make inference regarding the PRF.
name the assumptions of the classical linear regerssion model
elaborate on the “xt is non-stochastic or fixed in repeated samples” assumption
It is also referred to as “the fixed regressor assumption”.
What is it?
We have a list of sample values for the X variable(s). These are fixed and known. Under this assumption, if we re-draw the samples, the x-values remain the same.
However, the y-values are not necessarily the same. The reason being that the error term introduce randomness to the process due to various reasons.
Why do we bother?
It simplifies the concept because now, the y-values change as a result of the randomness from the error term only. If we do not make the fixed regressor assumption, we would also have randomness from the sampling variation.
We can also say that: Randomness from the error term/residual cause vertical movement of the y-values. Randomness from samples cause horizontal movement.
Whether the assumption of fixed regressor is reasonable or not depends on how the sample aligns with the population properties.
elaborate on consistency
Consistency is a large sample property.
A consistent estimator will approach the true value when the number of samples approach infinity.
Consistency is often considered to be the most important property of an estimator. This is due to how an estimator that is inconsistent cannot be trusted even with infinite data.
what is “required” for an unbiased estimator to not be consistent?
If its variance increase as the number of samples increase
what is standard error
Standard deviaiton of the statistic.
Recall that a statistic is a function of one or more random variables that takes the shape of a sample
what hypotheses do we have and their purposes
the null hyp
the alternative hyp
The null hyp is the statement that is actually being tested.
The alternative hypothesis represent the remaining cases.
what are the two ways to conduct a test
test of significance
confidence interval approach
what assumption must be made on the CLRM in order to make hypothesis tests?
The assumption that the error residuals are normally distributed.
elaborate on degrees of freedom
n the case of sample variance: if we have n sample points, meaning we have n random variables, which would be n degrees of freedom. We then compute the sample mean using the n degrees of freedom. This creates a constraint by adding ONE equality. Specifically, this equality say that when we insert values for n-1 of the sample variables, and use the computed value for the sample mean, we will get one final sample variable that must have a fixed value. This variable is no longer free to vary. because of this, we have lost one degree of freedom.
When we loose a degree of freedom, it is because there exist a dependency among the variables so that they are not completely independent. The mean thing is an example of a dependency among the variables.
Consider the regular variance formula. It use 1/n as the factor. When we take expectation of it, we get sigma squared as the result. Not surprising since we are computing the variance. This result is with n degrees of freedom.
if we instead use the sample mean estimator as a replacement for the true population mean, we impose a constraint, a dependency among the variables. We loose one degree of freedom.
When we loose a degree of freedom, we reduce the randomness of the model. Whetehr this has an impact on the variance of the model, and in what direciton, depends on the statistic.
Consider the sample mean estimator.
It has expectation equal to the true mean regardless of the degrees of freedom.
it has variance equal to sigma^2 /n.
In other words, fewer degrees of freedom creates larger variance.
The key is that introducing less degrees of freedom is about introducing dependencies among otherwise independent variables. Whether this cause the variance to be better or worse depends on the context and the statistic. However, this doesnt matter, as what we want is to estimate the true parameter. Therefore, what we need to understand is the degrees of freedom can fuck up this, so we need to make sure that our estimators works as epxected.
why does standardization work the way it does
first it is the simple shift in values.
Then it is the scaling.
The aim is that Var(something) = 1
Var(factor(X-u)) = 1
factor^2 Var(X-u) = 1
factor^2 Var(X) = 1
factor^2 sigma^2 = 1
sigma is fixed, we can only change factor. what must factor be to create the value 1.
Answer: it must be the inverse of sigma, 1/sigma.
This gives us:
Z = (X-u)/sigma
