SESSION 9 - CORRELATION AND SIMPLE LINEAR REGRESSION

Question 1

X and Y associated ? - What do you do first?

Answer 1

Scatter plots and Correlation

Question 2

Predict Y from X? - What do you do first?

Answer 2

Simple Linear Regression

Question 3

What do scatterplots do?

What do you look out for?

Answer 3

Shows relationship between 2 continuous variables

Linear association?
Outliers?

Question 4

What is correlation?

Answer 4

Correlation is a statistical measure that expresses the extent to which two variables are linearly related (meaning they change together at a constant rate).

Question 5

What is a simple linear regression?

Answer 5

Simple linear regression is used to estimate the relationship between two quantitative variables. You can use simple linear regression when you want to know:

How strong the relationship is between two variables (e.g., the relationship between rainfall and soil erosion).
The value of the dependent variable at a certain value of the independent variable (e.g., the amount of soil erosion at a certain level of rainfall)

Question 6

What are regression models?

Answer 6

Regression models describe the relationship between variables by fitting a line to the observed data. Linear regression models use a straight line, while logistic and nonlinear regression models use a curved line. Regression allows you to estimate how a dependent variable changes as the independent variable(s) change.

Question 7

What is the regression coefficient?

Answer 7

How much we expect Y to change as X increases.

Question 8

What is the E in the equation?

Answer 8

E is the error of the estimate, or how much variation there is in our estimate of the regression coefficient.

Question 9

What is Y?

Answer 9

Y is the predicted value of the dependent variable (y) for any given value of the independent variable (x).

Question 10

What is B0?

Answer 10

B0 is the intercept, the predicted value of y when the x is 0.

Question 11

What is B1?

Answer 11

B1 is the regression coefficient – how much we expect y to change as x increases.

Question 12

What is X?

Answer 12

X is the independent variable ( the variable we expect is influencing y).

Question 13

What is the Correlation Coefficient (r)?

Answer 13

The sample correlation coefficient (r) is a measure of the closeness of association of the points in a scatter plot to a linear regression line based on those points.

Possible values of the correlation coefficient range from -1 to +1, with -1 indicating a perfectly linear negative, i.e., inverse, correlation (sloping downward) and +1 indicating a perfectly linear positive correlation (sloping upward).

A correlation coefficient close to 0 suggests little, if any, correlation.

Question 14

What is Pearson Product-Moment Correlation?

Answer 14

The Pearson product-moment correlation coefficient (or Pearson correlation coefficient, for short) is a measure of the strength of a linear association between two variables and is denoted by r. Basically, a Pearson product-moment correlation attempts to draw a line of best fit through the data of two variables, and the Pearson correlation coefficient, r, indicates how far away all these data points are to this line of best fit (i.e., how well the data points fit this new model/line of best fit).

Question 15

How can we determine the strength of association based on the Pearson correlation coefficient?

Answer 15

The stronger the association of the two variables, the closer the Pearson correlation coefficient, r, will be to either +1 or -1 depending on whether the relationship is positive or negative, respectively. Achieving a value of +1 or -1 means that all your data points are included on the line of best fit – there are no data points that show any variation away from this line. Values for r between +1 and -1 (for example, r = 0.8 or -0.4) indicate that there is variation around the line of best fit. The closer the value of r to 0 the greater the variation around the line of best fit.

Question 16

What is the line of best fit?

Answer 16

It is also known as regression line

Line of best fit refers to a line through a scatter plot of data points that best expresses the relationship between those points. Statisticians typically use the least squares method (sometimes known as ordinary least squares, or OLS) to arrive at the geometric equation for the line, either through manual calculations or by using software.

Question 17

What is the spearmann’s rank correlation?

Answer 17

The Spearman’s rank-order correlation is the nonparametric version of the Pearson product-moment correlation. Spearman’s correlation coefficient, (ρ, also signified by rs) measures the strength and direction of association between two ranked variables.

Question 18

What are the assumptions of Spearman’s rank-order correlation?

Answer 18

You need two variables that are either ordinal, interval or ratio

Although you would normally hope to use a Pearson product-moment correlation on interval or ratio data, the Spearman correlation can be used when the assumptions of the Pearson correlation are markedly violated. However, Spearman’s correlation determines the strength and direction of the monotonic relationship between your two variables rather than the strength and direction of the linear relationship between your two variables, which is what Pearson’s correlation determines.

Question 19

What is a monotonic relationship?

Answer 19

A monotonic relationship is a relationship that does one of the following: (1) as the value of one variable increases, so does the value of the other variable; or (2) as the value of one variable increases, the other variable value decreases.

Question 20

If spearman’s greater than Pearson’s?

Answer 20

(If ρ>r, relationship monotonic and not linear)

Question 21

Correlation:
Pearson’s r – value range

Answer 21

Correlation:
Pearson’s r – value range

Perfect
negative
association

Negative
association

No
association

Positive
association

Perfect
positive
association

r=-1 r<0 r≈0 r>0 r=1

Question 22

What is pearson’s correlation coefficient not?

Answer 22

It is not the slope

Question 23

What is the slope?

Answer 23

That is the regression coefficient

Question 24

If there is a big outliers, what should be done?

Answer 24

Consider using Spearman’s rank

Question 25

What is Hypothesis Testing on Correlation Based On?

Answer 25

Hypothesis test: H0: r=0 (no association) H1: r≠0

Question 26

What are the assumptions of the Correlation Pearson's r

Answer 26

* X and Y continuous * Independence of observations * Homoscedastic * For valid tests and CIs – Random sample of individuals – X & Y (approximately) normally distributed

Question 27

What does Homoscedasticity mean?

Answer 27

Dependent variable (Y) has similar amounts of variance across range of values for independent variable (X)

Question 28

What is simple regression's hypothesis test?

Answer 28

H0: b1=0, no association between X and Y: * Independent variable X not associated with Y (X does not help predict Y)

Question 29

What is the t-test statistic in a simple regression model?

Answer 29

In a simple linear regression, the t-test statistic indicates whether the slope of the population regression line is significantly different from zero The t-test statistic determines if the slope of the regression line is significantly different from zero. This helps determine if the x-variable is a useful predictor of the y-variable The t-test statistic is calculated using the formula \(t=\beta /s_{b}\), where \(\beta \) is the sample regression coefficient and \(s_{b}\) is the residual standard error. What it means A t-value of zero means the sample results are exactly the same as the null hypothesis. As the difference between the sample data and the null hypothesis increases, the absolute value of the t-value increases In regression analysis, t-statistics evaluate the significance of regression coefficients, indicating if a variable significantly contributes to the mode

Question 30

What are fitted values?

Answer 30

In regression analysis, fitted values are the predicted values of the outcome variable for a given set of data. They are also known as predicted values. Fitted values are calculated using the estimated regression line, which is the line that minimizes the distance between the predicted and actual scores. The regression line is also known as the "line of best fit"

Question 31

What are residuals?

Answer 31

In linear regression, the difference between the fitted values and the true values of the outcome variable is called the residual. The residual is positive if the point lies above the regression line, and negative if it lies below.

Question 32

What is the coefficient of determination?

Answer 32

It is a measure of goodness of fit. A summary that indicates how well in the regression the independent variable explains the variation in the dependent variable Therefore, the coefficient of determination measures the fraction of the total variance in the dependent variable explained by the independent variable.