CHAPTER 5 Regression for Describing and Forecasting

Question 1

What is regression?

Answer 1

Finding the line of best fit through data to describe the relationship between two or more variables.

Question 2

What are the two primary uses of regression?

Answer 2

• Description
• Forecasting

Question 3

What is overfitting in regression?

Answer 3

A problem that arises when a model is too complex and fits the noise in the data rather than the underlying relationship.

Question 4

What does a regression equation express?

Answer 4

A linear relationship between a dependent variable and an independent variable.

Question 5

What are the regression parameters in a regression equation?

Answer 5

• α (intercept)
• β (slope)

Question 6

What does the intercept (α) represent in regression?

Answer 6

The predicted number of crimes on a day when the average temperature is 0 degrees Fahrenheit.

Question 7

What does the slope (β) represent in regression?

Answer 7

The amount that predicted crime increases with each degree Fahrenheit.

Question 8

What is the sum of squared errors (SSE)?

Answer 8

A measure of how well a regression line fits the data, calculated by summing the squared errors of each observation.

Question 9

What is ordinary least squares (OLS) regression?

Answer 9

A method to find the values of α and β that minimize the sum of squared errors.

Question 10

What are OLS regression coefficients?

Answer 10

The values of the parameters α and β that minimize the sum of squared errors.

Question 11

How do we interpret the OLS regression line?

Answer 11

It estimates the average number of crimes based on temperature and shows how crime changes with temperature.

Question 12

True or False: The regression line is always the best fit for all types of data.

Answer 12

False

Question 13

What is a conditional mean function?

Answer 13

A function that tells you the mean of some variable conditional on the value of another variable.

Question 14

What is the relationship between the regression line and conditional means?

Answer 14

The regression line is the best linear approximation to the conditional means.

Question 15

What would you minimize to find the conditional median?

Answer 15

The sum of the absolute values of the errors.

Question 16

What was a historical reason for using the sum of squared errors?

Answer 16

It is computationally easier to calculate using linear algebra.

Question 17

What does the regression line tell us about crime and temperature?

Answer 17

It predicts the average number of crimes based on temperature.

Question 18

What does it mean to regress crime on temperature?

Answer 18

To run an ordinary least squares regression where crime is the dependent variable and temperature is the independent variable.

Question 19

What is the significance of the slope of the regression line?

Answer 19

It indicates how much crime changes as the temperature changes.

Question 20

Fill in the blank: The line of best fit minimizes the _______.

Answer 20

sum of squared errors

Question 21

What is the graphical representation of errors in regression?

Answer 21

Vertical lines from data points to the regression line.

Question 22

In the context of regression, what does ‘parsimonious summary’ mean?

Answer 22

A concise representation of the data using the least number of parameters.

Question 23

What happens if we use an arbitrary line in regression?

Answer 23

It will yield poor forecasts.

Question 24

What is the role of statistical software in OLS regression?

Answer 24

To calculate the values of α and β that minimize the sum of squared errors.

Question 25

What data was used to illustrate regression in this chapter?

Answer 25

Crime and temperature data in Chicago.

Question 26

What is the importance of the slope in the context of crime data?

Answer 26

It quantifies the increase in crime for each degree increase in temperature.

Question 27

What does the conditional mean function graph represent?

Answer 27

The average number of crimes for each temperature bin.

Question 28

What is one alternative to using the sum of squared errors?

Answer 28

Minimizing the sum of absolute values of the errors.

Question 29

What does OLS stand for in the context of regression?

Answer 29

Ordinary Least Squares.

Question 30

What does the intercept (α O L S) in a regression equation represent?

Answer 30

The predicted turnout rate when the independent variable is zero.

Question 31

What is the predicted turnout rate for people aged zero according to the regression?

Answer 31

Approximately −14 percent.

Question 32

What does the slope (β O L S) in a regression equation indicate?

Answer 32

The average change in turnout rate for each additional year of age.

Question 33

What is the average increase in voter turnout for each additional year of age between 19 and 68?

Answer 33

Just over 1 percentage point.

Question 34

How much more likely are 68-year-olds to vote compared to 18-year-olds?

Answer 34

Approximately 50 percentage points more likely.

Question 35

What is the turnout rate for 18-year-olds in the 2014 election?

Answer 35

Approximately 4.8 percent.

Question 36

What are the predicted turnout rates for 69- and 70-year-olds?

Answer 36

Approximately 57.3 percent and 58.3 percent, respectively.

Question 37

True or False: The regression line can accurately predict turnout rates for infants.

Answer 37

False.

Question 38

What is one method to account for non-linearity in a regression analysis?

Answer 38

Fit separate regression lines for different segments of the data.

Question 39

What is one limitation of using a single linear regression for non-linear data?

Answer 39

It may produce a lot of error since it doesn't capture the true relationship.

Question 40

What is a polynomial regression?

Answer 40

A regression that includes powers of the explanatory variable, such as age-squared or age-cubed.

Question 41

What happens to the fit of the regression as more explanatory variables are added?

Answer 41

The fit typically improves, but it may lead to overfitting.

Question 42

What is overfitting in the context of regression analysis?

Answer 42

When a model becomes too complex and captures noise rather than the underlying relationship.

Question 43

What is the trade-off when adding more explanatory variables to a regression?

Answer 43

Increased complexity may lead to worse out-of-sample predictions.

Question 44

What is one reason why the tenth-order polynomial regression might perform poorly?

Answer 44

It captures meaningless correlations that do not hold in different data sets.

Question 45

What is the typical accuracy of polling averages in predicting presidential election outcomes?

Answer 45

Within 1 or 2 percentage points of the final vote share.

Question 46

Who is Nate Silver?

Answer 46

A journalist known for political data analytics and averaging polls.

Question 47

Fill in the blank: The relationship between age and voter turnout is approximately ______ from 18 to 68.

Answer 47

linear.

Question 48

What is the average accuracy of polls on Election Day?

Answer 48

Within 1 or 2 percentage points of the final vote share.

Question 49

Who established himself as a giant of political data analytics?

Answer 49

Nate Silver.

Question 50

True or False: The Electoral College allows some candidates to win the election while losing the popular vote.

Answer 50

True.

Question 51

What journal typically publishes a symposium before each presidential election?

Answer 51

PS: Political Science & Politics.

Question 52

What are some fundamental variables that might help predict vote share?

Answer 52

* Economic growth * Incumbency status * Number of war casualties

Question 53

What does the r² statistic represent in regression analysis?

Answer 53

The proportion of variance in the dependent variable that can be explained by the independent variables.

Question 54

What was the average prediction error of the regression using ten independent variables?

Answer 54

1.7 percentage points.

Question 55

What happens to the average prediction error when out-of-sample testing is applied?

Answer 55

It jumps up from 1.7 to 5.6 percentage points.

Question 56

Fill in the blank: A naive prediction based on a simple average of past elections gets within ______ percentage points.

Answer 56

4.6.

Question 57

What is overfitting in regression analysis?

Answer 57

Predicting a dependent variable with too many independent variables.

Question 58

What is the purpose of out-of-sample testing?

Answer 58

To assess the predictive accuracy of a model using data not included in the original regression.

Question 59

What is the intercept in a regression context?

Answer 59

The predicted value of the outcome when all explanatory variables are set to 0.

Question 60

What is the sum of squared errors (SSE)?

Answer 60

The total of squared differences between actual and predicted values.

Question 61

Who is credited with coining the term 'regression'?

Answer 61

Francis Galton.

Question 62

What phenomenon did Galton discover related to regression?

Answer 62

Regression to the mean.

Question 63

What does the slope of the regression line indicate?

Answer 63

The sign and magnitude of the relationship between two variables.

Question 64

What is the common form for presenting regression results?

Answer 64

In a table.

Question 65

What is the conditional mean function?

Answer 65

A function that tells the mean of a variable given the value of other variables.

Question 66

What type of regression uses the method of ordinary least squares?

Answer 66

OLS regression.

Question 67

What is the root mean squared error (Root-MSE)?

Answer 67

The square root of the mean squared error, indicating average prediction deviation.

Question 68

What is an independent or explanatory variable?

Answer 68

A variable used to predict or explain the dependent variable.

Question 69

What is a dependent variable?

Answer 69

The variable associated with the outcome we are trying to describe, predict, or explain.

Question 70

What might cause random variables to appear correlated with outcomes in regression?

Answer 70

Chance.

Question 71

What did Galton's analysis of heights reveal?

Answer 71

Sons tend to be taller than their fathers but shorter than average.

Question 72

What is the goal of researchers when trying to predict election outcomes?

Answer 72

To find new variables to improve predictive power.

Question 73

What is the conditional mean function?

Answer 73

A function that tells you the mean (average) of some variable conditional on the value of some other variables.

Question 74

Define out-of-sample prediction.

Answer 74

Using regression (or another statistical technique) to predict the outcome for observations that were not included in the original data you used to generate your predictions.

Question 75

What is overfitting?

Answer 75

Attempting to predict a dependent variable with too many independent variables, so that variables appear to predict the dependent variable in the data but have no actual relationship with it in the world.

Question 76

What does the dataset SchoolingEarnings.csv provide?

Answer 76

The average annual earnings for 41- to 50-year-old men in the United States in 1980 at each level of schooling.

Question 77

What is the dependent variable in the regression exercise discussed?

Answer 77

Earnings.

Question 78

Fill in the blank: A _______ is a method to predict earnings using only years of schooling.

Answer 78

parsimonious way

Question 79

What is the first step in examining the relationship between earnings and schooling?

Answer 79

Start by making a scatter plot.

Question 80

What type of regression should be run to include schooling, schooling^2, schooling^3, and schooling^4?

Answer 80

Fourth-order polynomial regression.

Question 81

What is the purpose of running different regressions for some different ranges of schooling?

Answer 81

To see if those lines look meaningfully different from the predictions you get from a single regression including all the data.

Question 82

Does all the analysis make you think the simple linear approach was reasonable or unreasonable?

Answer 82

Subjective judgment based on analysis results.

Question 83

What is the goal of conducting out-of-sample tests?

Answer 83

To evaluate your prediction strategy.

Question 84

Using only data for those with twelve years of schooling or less, what should you predict?

Answer 84

Earnings for those with more than twelve years of schooling.

Question 85

Who wrote 'The History of Statistics: The Measurement of Uncertainty before 1900'?

Answer 85

Stephen M. Stigler.

Question 86

True or False: Asking enough people to guess the weight of an ox will always yield a correct answer.

Answer 86

False.

Question 87

What is the principle behind asking many non-experts for their guesses?

Answer 87

Their errors will cancel out and you’ll get a good answer.