flashcard 7

Question 1

What is a statistical model, and why are regression models commonly used?

Answer 1

A statistical model is a simplified representation of reality that characterizes the association structure in data. Regression models are common because they explicitly model how one or more predictor (independent) variables relate to an outcome (dependent) variable, allowing quantification of those relationships.

Question 2

How does simple linear regression differ from multiple linear regression?

Answer 2

Simple linear regression uses one predictor variable to model the outcome as a linear function, whereas multiple linear regression includes two or more predictors, still assuming a linear relationship between each predictor and the outcome.

Question 3

What key assumption underlies the use of linear regression regarding the relationship between variables?

Answer 3

The primary assumption is that the expected value of the outcome (Y) is a linear function of the predictor(s) (X), meaning that changes in X correspond to proportional changes in Y on average.

Question 4

In linear regression, what do the intercept (α) and slope (β) represent?

Answer 4

The intercept (α) represents the expected value of the outcome when all predictors equal zero. Each slope (β) represents the expected change in the outcome for a one-unit increase in the corresponding predictor, holding other predictors constant.

Question 5

Why is it important that the error terms (ε) in linear regression are independent and identically distributed with mean zero?

Answer 5

Independence ensures that errors for different observations do not influence each other, and identically distributed with mean zero implies that deviations around the fitted line are random, unbiased, and have constant variance; violating these can lead to incorrect inferences.

Question 6

What does homoscedasticity mean in the context of linear regression?

Answer 6

Homoscedasticity means that the variance of the error terms (residuals) remains constant across all levels of the predictor variables. In other words, the spread of residuals does not systematically increase or decrease as fitted values change.

Question 7

Why must predictor variables in a multiple linear regression not be highly correlated with one another?

Answer 7

High correlation among predictors (multicollinearity) inflates the variance of coefficient estimates, making them unstable and difficult to interpret, and can obscure the unique contribution of each predictor.

Question 8

How is the coefficient of determination (R²) interpreted in linear regression?

Answer 8

R² represents the proportion of total variability in the outcome that is explained by the fitted regression model.

For example, R² = 0.60 means 60% of the outcome’s variance is accounted for by the predictors.

Question 9

What purpose does the adjusted R² serve compared to the ordinary R²?

Answer 9

Adjusted R² penalizes R² for adding predictors that do not meaningfully improve model fit. It adjusts for the number of predictors relative to sample size, helping to prevent overfitting by reflecting model complexity.

Question 10

Why can linear regression be seen as a generalization of the t-test and ANOVA?

Answer 10

A t-test compares the means of two groups, and ANOVA compares means across multiple groups. In linear regression, categorical group membership can be encoded as indicator variables, so a regression model with a group indicator tests the same mean differences.

Question 11

What is the role of the standard error (SE) of a regression coefficient?

Answer 11

The standard error measures how precisely the regression coefficient is estimated, reflecting the variability of the estimate across hypothetical repeated samples. Smaller SE indicates more precise estimation.

Question 12

How do you use a t-test within linear regression to assess whether a coefficient is statistically significant?

Answer 12

You compute a t-statistic as (β / SE(β)) and compare it to the appropriate t-distribution. A large magnitude of t (and corresponding low p-value) indicates the coefficient differs significantly from zero, suggesting a meaningful association.

Question 13

Describe how to interpret a 95% confidence interval (CI) for a regression coefficient.

Answer 13

A 95% CI is the range that, under repeated sampling, would contain the true coefficient 95% of the time. If the CI does not include zero, it indicates a statistically significant association at the 5% level.

Question 14

Why is the value 1.96 used when constructing a 95% CI for a normally distributed estimate?

Answer 14

In a standard normal distribution, approximately 95% of values lie within ±1.96 standard deviations from the mean. Thus, multiplying SE by 1.96 produces the bounds for a 95% CI.

Question 15

How can a standard curve (calibration curve) in analytical chemistry rely on linear regression?

Answer 15

A standard curve is formed by measuring a known analyte’s signal (e.g., absorbance) at various concentrations. Linear regression fits a straight line relating concentration to signal. Unknown samples’ signals are then placed on the fitted line to estimate their concentrations.

Question 16

What distinguishes logistic regression from linear regression in terms of the outcome variable?

Answer 16

Logistic regression models a binary or categorical outcome (e.g., disease yes/no), whereas linear regression models a continuous outcome. Logistic regression uses the logit (log-odds) link to ensure predicted probabilities range between 0 and 1.

Question 17

In logistic regression, what does the logit function (log-odds) accomplish?

Answer 17

The logit function transforms a probability p∈(0,1) to the entire real line (–∞, +∞) via log(p / (1–p)). This linearizes the relationship between predictors and log-odds, allowing fitting with standard linear model techniques.

Question 18

How is an odds ratio (OR) derived from a logistic regression coefficient (β)?

Answer 18

The odds ratio for a one-unit increase in a predictor is exp(β). If β = 0.5, then OR = e⁰·⁵ ≈ 1.65, meaning the odds of the outcome are 1.65 times higher per unit increase in that predictor.

Question 19

When is an odds ratio (OR) considered statistically significant based on its 95% CI?

Answer 19

If the 95% CI for the OR does not include 1, the association is statistically significant at the 5% level. Inclusion of 1 implies no significant difference in odds between comparison groups.

Question 20

Why are odds ratios used in retrospective (case–control) studies instead of risk ratios?

Answer 20

In case–control studies, the total number of exposed individuals in the population is unknown, so absolute risks cannot be calculated. However, odds within cases and controls are observable, allowing OR computation.

Question 21

Under what conditions can risk ratios (RR) be used instead of odds ratios (OR)?

Answer 21

Risk ratios can be calculated in prospective (cohort) studies where both the number of events and total individuals at risk in each exposure group are known, allowing direct computation of incidence proportions.

Question 22

How do odds ratios and risk ratios relate when the event of interest is rare?

Answer 22

When the event is rare (i.e., incidence is very low), the numerical values of OR and RR are very close because odds and probabilities converge when probabilities are small.

Question 23

What key assumptions are shared by both linear and logistic regression models?

Answer 23

Both models assume that observations are independent, that predictors are measured without error, and that the specified link function (identity for linear, logit for logistic) correctly describes how predictors relate to the transformed outcome.

Question 24

Why is it incorrect to infer causation solely from an observed association in regression or correlation analyses?

Answer 24

Regression and correlation reveal statistical association but do not control for unmeasured confounders or establish temporal precedence. Establishing causation requires controlled experimental design, such as randomized controlled trials, to rule out alternative explanations.

Question 25

How does multicollinearity affect the interpretation of logistic regression coefficients?

Answer 25

Multicollinearity (high correlation between predictors) inflates the standard errors of logistic regression coefficients, making OR estimates unstable and reducing confidence in interpreting any single predictor’s effect.

Question 26

Explain why logistic regression uses maximum likelihood estimation rather than ordinary least squares.

Answer 26

Because the outcome is binary, residuals are not normally distributed around a continuous response. Maximum likelihood estimation finds the parameter values that maximize the probability of observing the given binary outcomes under the logistic model.

Question 27

What is the practical difference between a probability and an odds when interpreting logistic regression?

Answer 27

Probability is the chance of an event happening ([0,1] range). Odds are the ratio of the probability of the event to the probability of no event (p / (1–p)). Odds greater than 1 indicate p > 0.5, and vice versa.

Question 28

How do confidence intervals around odds ratios inform the reliability of estimated associations?

Answer 28

The CI width reflects uncertainty: a narrow CI indicates a precise OR estimate, while a wide CI indicates less precision. If the entire CI lies above (or below) 1, it confirms a reliably increased (or decreased) odds of the outcome.

Question 29

Why might logistic regression include continuous and categorical predictors simultaneously?

Answer 29

Logistic regression can model a binary outcome as a function of any mix of predictor types, allowing adjustment for confounding variables, capturing nuanced relationships, and improving predictive accuracy by using all relevant information.

Question 30

What does an R²-like measure (e.g., pseudo-R²) indicate in logistic regression?

Answer 30

Pseudo-R² measures (such as McFadden’s R²) quantify how well the logistic model explains outcome variability in a way analogous to linear R². Although they do not have exactly the same interpretation, higher values indicate better model fit.

Question 31

How does one interpret a regression coefficient of zero in both linear and logistic regression?

Answer 31

In linear regression, a coefficient of zero implies no change in the continuous outcome per unit change in predictor (no association). In logistic regression, a coefficient of zero implies an OR of 1 (no change in odds of the outcome), similarly indicating no association.

Question 32

When constructing a standard curve via linear regression, why must one check that the relationship between concentration and signal is approximately linear?

Answer 32

Because linear regression assumes a linear relationship; if the calibration points do not align linearly (e.g., at very high concentrations saturation occurs), the fitted line will not accurately predict unknown concentrations, leading to biased estimates.

Question 33

Why is the concept “association does not imply causation” emphasized when teaching regression and correlation?

Answer 33

Because it reminds practitioners that observed statistical relationships might arise from confounding variables, reverse causality, or chance, and that only controlled experimental designs can rigorously establish a causal link.

Question 34

How can one use the fitted logistic regression model to predict an individual’s probability of an outcome?

Answer 34

First calculate the linear predictor (η = α + β₁ X₁ + … + βₖ Xₖ), then transform via the logistic function: probability = 1 / [1 + exp(–η)]. This yields a value between 0 and 1 representing the predicted likelihood of the event.

Question 35

Explain why one would examine confidence intervals of regression coefficients rather than rely solely on p-values.

Answer 35

Confidence intervals convey both statistical significance (via inclusion/exclusion of a null value) and the magnitude and precision of effect estimates. P-values alone do not show how large or precise an effect is, only whether it is unlikely to be zero by chance.