statistics - topic 7 - bivariate regression

Question 1

what is regression analysis used to do ?

Answer 1

Explain the impact of changes in an independent variable on a dependent variable
Predict the value of a dependent variable based on the value of at least one independent variable

Question 2

what is the dependent variable?

Answer 2

The dependent variable is the variable we wish to explain (also called the endogenous variable)

Question 3

what is the independent variable?

Answer 3

The independent variable is the variable used to explain the dependent variable (also called the exogenous or explanatory variable)

Question 4

what does the population regression model show?

Answer 4

it shows the relationship between two variables

Question 5

what are the components of a population regression model?

Answer 5

it has a dependent variable, population intercept, population slope coefficient, independent variable and an error term

Question 6

what is the sample data used for ?

Answer 6

Sample data is used to provide an estimate of the population regression model

Question 7

what are the assumptions required for the least squares estimation to be an accurate estimate?

Answer 7

The true relationship is linear (𝑌 is a linear function of 𝑋, plus a random error)
The error term, 𝜀_𝑖, is uncorrelated with the random variable, 𝑋
The error term, 𝜀_𝑖, has a mean of 0 and constant variance, 𝜎^2 (the latter property is called homoscedasticity):
𝐸[𝜀_𝑖 ]=0 and 𝐸[𝜀_𝑖^2 ]=𝜎^2 for 𝑖=1,…,𝑛
The error terms, 𝜀_𝑖, are not correlated with one another, so that:
𝐸[𝜀_𝑖 𝜀_𝑗 ]=0 for all 𝑖≠𝑗

Question 8

what is the least squares method?

Answer 8

Least squares provides estimates of 𝛽_0 and 𝛽_1 by finding the values of 𝑏_0 and 𝑏_1 that minimize the sum of the squared errors (SSE):
min⁡𝑆𝑆𝐸=min⁡∑(𝑖=1)^𝑛▒𝑒_𝑖^2 =min⁡∑(𝑖=1)^𝑛▒(𝑦_𝑖−𝑦̂𝑖 )^2 =min⁡∑(𝑖=1)^𝑛▒[𝑦_𝑖−(𝑏_0+𝑏_1 𝑥_𝑖 )]^2

Question 9

what is the equation for b1 in the least squares coefficient estimator?

Answer 9

𝑏_1
=(∑(𝑥_𝑖−𝑥̅ )(𝑦_𝑖−𝑦̅ ) ) /(∑(𝑥_𝑖−𝑥̅ )^2 )
=𝐶𝑜𝑣(𝑥,𝑦)/(𝑠_𝑥^2 )
=𝑟 x (𝑠_𝑦/𝑠_𝑥 )
where 𝑟 is 𝐶𝑜𝑟𝑟(𝑥,𝑦)

Question 10

what is the regression line after you have estimated b1?

Answer 10

𝑏_0=𝑦̅−𝑏_1 𝑥̅
because the regression line goes through the sample means 𝑥̅, 𝑦̅

Question 11

what are the two parts of the variation in a dependent ratio?

Answer 11

the total sum of the squares is eqal to the regression sum of the squares + the error sum of the squares

Question 12

what is the formula for the regression sum of the squares?

Answer 12

∑(𝑦̂_𝑖−𝑦̅ )^2 where 𝑦̂_𝑖 = predicted value of the dependent variable given 𝑋=𝑥_𝑖 and 𝑦̅ = sample mean of the dependent variable

Question 13

what is the formula for the error sum of the squares?

Answer 13

∑(𝑦_𝑖−𝑦̂_𝑖 )^2 where 𝑦_𝑖 = observed value of the dependent variable and 𝑦̂_𝑖 = predicted value of the dependent variable given 𝑋=𝑥_𝑖

Question 14

what is the coefficient of determination?

Answer 14

it is the proportion of the total variation in the depenedent variable that is explained by variation in the independent variable

Question 15

what is the formula for the coefficient of determination?

Answer 15

𝑅^2=𝑆𝑆𝑅/𝑆𝑆𝑇=(𝑟𝑒𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛 𝑠𝑢𝑚 𝑜𝑓 𝑠𝑞𝑢𝑎𝑟𝑒𝑠)/(𝑡𝑜𝑡𝑎𝑙 𝑠𝑢𝑚 𝑜𝑓 𝑠𝑞𝑢𝑎𝑟𝑒𝑠)

Question 16

in what range does the coefficent of determination live in and what is it equal to?

Answer 16

0≤𝑅^2≤1 and 𝑅^2=𝑟^2 where 𝑟 denotes the correlation coefficient

Question 17

what does it mean when R^2 is equal to 1?

Answer 17

When 𝑅^2=1, there is a perfect linear relationship between 𝑋 and 𝑌: 100% of the variation in 𝑌 is explained by variation in 𝑋

Question 18

what does it mean when R^2 is equal to 0?

Answer 18

When 𝑅^2=0, there is no linear relationship between 𝑋 and 𝑌: the value of 𝑌 does not depend on 𝑋

Question 19

what is the standard deviation of e_i and why is this the case?

Answer 19

The standard deviation of 𝑒_𝑖 is:
𝑠_𝑒=√((∑(𝑒_𝑖^2 )/(𝑛−2))=√(𝑆𝑆𝐸/(𝑛−2))
Division is by 𝑛–2 instead of 𝑛–1 because the estimated regression model contains two estimated coefficients, 𝑏_0 and 𝑏_1

Question 20

what is the standard deviation for b1?

Answer 20

The standard deviation of 𝑏_1 is:
𝑠_(𝑏_1 )=√((𝑠_𝑒^2) / (∑(𝑥_𝑖−𝑥̅ )^2 ))

Question 21

what does the standard deviation of b1 show?

Answer 21

It is a measure of variation in the slope of regression lines from different samples

Question 22

what is the test statistic for a hypothesis test of a slope?

Answer 22

𝑡=𝑏_1/𝑠_(𝑏_1 )

Question 23

what is the decision rule for the hypothesis test of a slope?

Answer 23

Reject 𝐻_0 if 𝑡< −𝑡_(𝑛−2, 𝛼∕2) or 𝑡>𝑡_(𝑛−2,𝛼∕2)

Question 24

what is the formula for the confidence interval of a slope?

Answer 24

𝑏_1±𝑡_(𝑛−2,𝛼∕2) 𝑠_(𝑏_1 )

Question 25

what will be affected by the units of measurement used?

Answer 25

The units used to measure the dependent and independent variable will affect 𝑏_0, 𝑏_1, 𝑠_(𝑏_0 ), 𝑠_(𝑏_1 ) and the upper and lower confidence limits

Question 26

if you multiply the dependent variable by a what is the effect on the statistics?

Answer 26

it will be a times bigger

Question 27

if you multiply the independent variable by a what is the effect on the statistics?

Answer 27

it will be a times smaller

Question 28

what will not be affected by the units of measurement?

Answer 28

the test statistic and the p value will be unaffected

Question 29

what are outliers?

Answer 29

Outliers are observations that have 𝑌 values that differ substantially from their predicted value, 𝑦 ̂_𝑖

Question 30

what are extreme points?

Answer 30

Extreme points are observations that have 𝑋 values that differ substantially from the 𝑋 values of other observations

Question 31

what is the confidence interval for the mean value of Y, given a particular value of X,?

Answer 31

𝑦 ̂_(𝑛+1)±𝑡_(𝑛−2,𝛼∕2) 𝑠_𝑒 √(1/𝑛+(𝑥_(𝑛+1)−𝑥 ̅ )^2/∑(𝑥_𝑖−𝑥 ̅ ) ^2 )

Question 32

how does the size of the interval vary with the distance of Xn+1 from the mean x?

Answer 32

it varies postively

Question 33

what is the prediction interval for Yn+1 given a particular value of X ?

Answer 33

𝑦 ̂_(𝑛+1)±𝑡_(𝑛−2,𝛼∕2) 𝑠_𝑒 √(1+1/𝑛+(𝑥_(𝑛+1)−𝑥 ̅ )^2/∑(𝑥_𝑖−𝑥 ̅ ) ^2 )

Question 34

what is the point of the +1 term in the prediction interval for Yn+1?

Answer 34

it adds to the interval width to reflect the added uncertainty for an individual case