correlation and regression

Question 1

Pearson Bivariate Correlation Coefficient

Define and Calculate

Answer 1

it’s a number that shows how two things are related in a straight line (strength and direction linear relationship between two continuous variables)
calculate: It is calculated by dividing the covariance of the two variables by the product of their standard deviations.

Question 2

Pearson Bivariate Correlation Coefficient

range and interpretation

Answer 2

The coefficient ranges from -1 to +1, where -1 indicates a perfect negative linear relationship, +1 indicates a perfect positive linear relationship, and 0 indicates no linear relationship.

Question 3

Plotting Pearson

Answer 3

Linearity Assessment Plot: A plot used to assess the linearity of the relationship between two variables.
Helps determine if the relationship between variables is linear or nonlinear with a trend line.
Scatter plot is commonly used for this purpose.

points cluster around the trend line = stronger correlation

Question 4

Z-scores

defintion

Answer 4

Z scores are a way to standardize data points by showing how many standard deviations they are from the mean.

a z score of +2 indicates that a data point is two standard deviations above the mean, while a z score of -1 indicates that a data point is one standard deviation below the mean.

Question 5

Standardizing with Z Scores

meaning, proedure and purpose

Answer 5

Standardizing allows fair comparison of data on a common scale.
Meaning: It transforms data to have a mean of 0 and a standard deviation of 1.Standardizing converts data into z scores for fair comparison.
Procedure: Subtract the mean and divide by the standard deviation.
Purpose: Makes different data comparable by putting them on the same scale.

Question 6

Why Standardize and Meaning

Answer 6

Standardizing allows fair comparison of data on a common scale.
It transforms data to have a mean of 0 and a standard deviation of 1.

Question 7

Calculate Z-scores

Answer 7

Calculation:
(X- μ) / σ

X is the raw score, μ is the mean of the distribution, and is the standard deviation.

Question 8

Standard Deviation

Define

Answer 8

A measure of the amount of variation or dispersion in a set of values

It’s a measure of how spread out numbers are in a set

Question 9

Interpretation of Standard Deviation

Answer 9

A low standard deviation indicates that the data points tend to be close to the mean, while a high standard deviation indicates that the data points are spread out over a wider range of values.

Question 10

Why is it good for data to cluster around the mean?

Relaibility, comparability and predictability

Answer 10

Reliability: There are fewer extreme values or outliers that could skew the interpretation of the data.
Predictability: it makes it easier to predict future outcomes or estimate probabilities. This is because there is less uncertainty or variability in the data.
Comparability: When data points are spread out, it can be challenging to compare different groups or datasets, but when they are close to the mean, comparisons become more straightforward.

Question 11

Calculate Stanadard Deviaition

Answer 11

the square root of the variance

Question 12

Varinace

Define

Answer 12

A measure of how spread out or dispersed the values in a data set are from the mean.

Question 13

Calculate the Variance

Answer 13

taking the average of the squared differences between each data point and the mean

Question 14

interpretation of the variance

Answer 14

A larger variance indicates greater variability or dispersion in the data set, while a smaller variance suggests that the data points are closer to the mean.

e.g. If the variance of a set of test scores is 25, it means that, on average, each score differs from the mean by 25 squared units

Question 15

Using Z-Scores to Identify and Deal with Outliers

Answer 15

Standardization: Transforming data into z-scores with a mean of 0 and standard deviation of 1.
Thresholds: Outliers defined as z-scores beyond a certain threshold (e.g., z > 2 or z < -2).
Comparability: Allows fair comparison of outliers across datasets.
Data Cleaning: Outliers identified using z-scores can be examined for errors or significance.

Question 16

Correlation Coefficient (r)

Answer 16

Measures the strength and direction of the linear relationship between two variables.
Range: Between -1 and +1, where -1 indicates a perfect negative linear relationship, +1 indicates a perfect positive linear relationship, and 0 indicates no linear relationship.

Question 17

Interpreation of correlation coefficent

Answer 17

Magnitude: It shows how strong the relationship is. If |r| is closer to 1, the relationship is stronger. If it’s closer to 0, the relationship is weaker.

e.g. Correlation Coefficient (-0.42): Indicates a moderate negative linear relationship between the variables being studied.

Question 18

Significance Level (p-value)

Answer 18

It indicates the probability that the observed result (or more extreme) occurred by random chance, assuming the null hypothesis is true.

observed effect is likely to be genuine or random

Question 19

Interpreation of p-value

Answer 19

lower p-value suggests stronger evidence against the null hypothesis, indicating that the observed result is unlikely to be due to chance

Typically, a significance level of 0.05 (or 5%) is used. If the p-value is less than this threshold, the result is considered statistically significant

Question 20

Cohen’s Rules of Thumb for Magnitude of Correlation

Answer 20

Definition: Guidelines for interpreting the strength of correlation coefficients.
Small: Magnitude around 0.10.
Moderate: Magnitude around 0.30.
Large: Magnitude around 0.50.

-0.42 = moderate negative correlation

Question 21

Correlation Coefficient (0.15)

Answer 21

Since 0.15 is closer to 0.10, it would be considered a small correlation according to Cohen’s rules of thumb

Question 22

Correlation Coefficient (-0.60)

Answer 22

Since -0.60 is greater than 0.50, it would be considered a large negative correlation

Question 23

Correlation Coefficient (0.35)

Answer 23

Since 0.35 falls between 0.30 and 0.50, it would likely be considered a moderate positive correlation

Question 24

Correlation Coefficient (-0.25)

Answer 24

Since -0.25 falls between -0.10 and -0.30, it would likely be considered a small negative correlation.

Question 25

Simple Regression Model

Answer 25

A statistical technique used to model the relationship between one predictor variable and one outcome variable.
To understand how changes in the predictor variable are associated with changes in the outcome variable.

Question 26

Simple Regression

Answer 26

Examines how changes in one variable (predictor) are associated with changes in another variable (outcome).
Used to predict the value of the outcome variable based on the value of the predictor variable.
Provides an equation that describes the relationship and allows for making predictions.
Can suggest causality if appropriate conditions are met, as it implies a directional relationship between variables

Question 27

Regression Equation

Answer 27

Y = bX+ c+ e

Question 28

Regression Equation Explained

Answer 28

Y represents the outcome variable (e.g., test scores).
X represents the predictor variable (e.g., study time).
b is the regression coefficient, which indicates the change in Y for a one-unit change in X.
c is the intercept, representing the value of Y when X is zero
e is the error term, representing the difference between the observed Y and the predicted Y based on the regression equation.

Question 29

Regression Model in an example: attention span = b(screen time) + c + e

what is b, c and e

Answer 29

The outcome variable is “attention span”.
The predictor variable is “screen time”.
b is the regression coefficient representing the effect of screen time on attention span.
c is the intercept representing the value of attention span when screen time is zero.
e is the error term representing unexplained variability in attention span.

Question 30

R-squared

Answer 30

It shows how well the predictor variable(s) explain the outcome variable in a regression model.

Question 31

Range of R-saqured

What is an R-squared value of 0.40 mean?

Answer 31

Between 0 and 1, where 0 means no explanation and 1 means full explanation.

40% of the outcome variation is explained by the predictor

Question 32

R-squared Relation to Correlation Coefficient

Answer 32

R-squared equals the square of the correlation coefficient.

Example: If the correlation coefficient is -0.42, the R-squared would be
(−0.42)^2 = 0.1764
17.64%

Question 33

Multiple regression

Answer 33

Statistical technique used to examine the relationship between one dependent variable and two or more independent variables
assumptions include linearity, independence of errors, homoscedasticity, normality

Question 34

How do you interpret the regression coefficients in multiple regression?

Answer 34

Regression coefficients represent the change in the dependent variable for a one-unit change in the predictor variable, holding other variables constant.

Question 35

How do you assess the overall fit of a multiple regression model

Answer 35

The overall fit can be assessed using measures such as R-squared and adjusted R-squared, which indicate the proportion of variance explained by the model.

Question 36

Importance of Checking Scatterplot before Reporting Correlation Coefficient

Answer 36

To visually inspect data and validate assumptions before interpreting correlation coefficient.
Allows visual inspection of assumptions.
Helps detect linearity and outliers.
Ensures accurate interpretation of correlation.

Before reporting a correlation coefficient between two variables, examining a scatterplot allows you to identify any nonlinear relationships or outlier points that may affect the interpretation of the correlation.

Question 37

Third Variable Problem in Correlation

Answer 37

The presence of a third variable that influences both variables being correlated, leading to a spurious or misleading correlation.

Question 38

How to mitigate Third Variable Problem in Correlation

Answer 38

Control for or consider potential third variables to accurately interpret the correlation between two variables. Use techniques like partial correlation or regression analysis to account for the influence of third variables.

Question 39

Partial Correlation

Answer 39

A statistical technique used to assess the relationship between two variables while controlling for the effects of one or more additional variables.

Calculates the correlation coefficient between two variables after statistically removing the influence of one or more covariates.

Question 40

Regression Analysis

Answer 40

A statistical method that examines the relationship between one dependent variable and one or more independent variables.

Identifies how changes in the independent variables are associated with changes in the dependent variable.

Fits a regression model to the data, estimating the coefficients that represent the strength and direction of the relationships between variables.

Question 41

Difference between Positive and Negative Correlation

Answer 41

Positive: Both variables increase together (e.g., height and weight).Negative: One variable increases, the other decreases (e.g., alcohol consumption and memory recall).

Question 42

Checking for Influence of Outliers on Correlation Coefficient

Answer 42

1) Plot the data using a scatterplot to visually identify outliers.
2) Calculate the correlation coefficient with and without outliers to observe changes in its magnitude.
3) Conduct sensitivity analyses by removing outliers and re-calculating the correlation coefficient.

If the correlation coefficient changes substantially after removing outliers, it suggests that outliers may have influenced the correlation.

Question 43

Cohen’s Criteria for Small, Medium, and Large Correlation Coefficients

Answer 43

Small: 0.1
Medium: 0.3
Large: 0.5

Question 44

interpretation of a confidence interval for a correlation coefficient

Answer 44

If the confidence interval includes zero, the correlation coefficient is not statistically significant at the specified confidence level.
If the confidence interval does not include zero, the correlation coefficient is statistically significant at the specified confidence level.
When we say “includes zero,” we mean the entire interval falls on one side of zero.

Question 45

Confidence Interval

Answer 45

When we calculate a correlation coefficient between two variables, we also calculate something called a confidence interval.
The confidence interval tells us a range of values within which we are reasonably confident the true correlation lies.

Question 46

A Correlation coefficient of 0.50 with a 95% confidence interval of (0.30, 0.70)

Answer 46

Correlation coefficient of 0.50 with a 95% confidence interval of (0.30, 0.70) indicates that we are 95% confident that the true correlation lies between 0.30 and 0.70.

Question 47

X and Y

Components of Linear Regression Model

Answer 47

Dependent Variable (Y): The variable being predicted or explained by the independent variables.
Independent Variables (X): The variables used to predict or explain changes in the dependent variable.

Question 48

Regression Coefficients (β)

Components of Linear Regression Model

Answer 48

The parameters representing the relationship between each independent variable and the dependent variable.

Question 49

Intercept (β₀)

Components of Linear Regression Model

Answer 49

The constant term in the regression equation, representing the value of the dependent variable when all independent variables are zero.

Components of Linear Regression Model

Question 50

Residuals (ε)

Components of Linear Regression Model

Answer 50

The differences between the observed and predicted values of the dependent variable.

Question 51

Error Term (ε)

Components of Linear Regression Model

Answer 51

Represents the variability in the dependent variable that cannot be explained by the independent variables.

Question 52

Correlation Matrix

Answer 52

A table that shows the correlation coefficients between multiple variables in a dataset.

Square matrix where each row and column represents a variable.
The cells contain correlation coefficients, showing the strength and direction of relationships between variables.

Question 53

Interpretation of Correlation Matrix

Answer 53

Values range from -1 to 1.
Positive values indicate a positive correlation (variables move in the same direction).
Negative values indicate a negative correlation (variables move in opposite directions).
Values closer to 1 or -1 represent stronger correlations, while values closer to 0 represent weaker correlations.

Question 54

Coefficients

Slope (b) and intercept (c)

Answer 54

Slope (b): Represents the change in the dependent variable for a one-unit change in the independent variable.
Intercept (c): Represents the predicted value of the dependent variable when all independent variables are zero.