Logistic regression

Question 1

what is the difference between Logistic Regression and Generative Classifiers (LDA, QDA, Naive Bayes):

Answer 1

Generative Classifiers (LDA, QDA, Naive Bayes) - These classifiers take a different approach. Instead of directly estimating the probability of passing or failing, they try to understand how the data was generated. They learn the distribution of features (like the number of hours studied) separately for passing and failing students. Then, they use this information to estimate the probability of passing or failing for any given number of hours studied.

Difference:

Logistic regression focuses on directly estimating the probability of passing or failing based on the input features (like hours studied).

Generative classifiers try to understand how the data was generated and use that information to make predictions.

Conclusion:

Logistic regression is a straightforward and powerful technique for predicting probabilities directly.
Generative classifiers take a more complex approach by modeling the underlying data distribution, which can sometimes lead to better performance when the data is well understood and follows certain patterns.

Question 2

Logistic Regression

Answer 2

Logistic Regression is part of broad class of methods known as generalised liner models(GLM). so they take liner model, and generalize by essentially having the link function between the liner predictor and the outcome. outcome don’t need to be normally distributed , it may be distributed using some completely separate distribution.
there’s a link function that ties the the mean and the variance of that distribution uh to the linear predictor so logistic regression models a binary response

Question 3

Logistic and logit curves

Answer 3

“Logistic” and “logit” are closely related terms commonly used in logistic regression, a statistical method for modeling binary outcomes.
Logistic Function: The logistic function, also known as the sigmoid function, is a mathematical function that maps any real-valued number to a value between 0 and 1.
In logistic regression, this function is used to model the probability that a given observation belongs to a particular category (usually coded as 1 for one category and 0 for the other). The output of the logistic function represents the probability of the positive outcome (e.g., the probability of an event occurring).

Logit Function: The logit function is the inverse of the logistic function.In logistic regression, the logit function is used to transform the probability of the positive outcome (p) into a linear relationship with the predictor variables. The logit transformation maps the probability values onto the entire real number line, allowing for a linear relationship between the predictors and the log-odds of the outcome.

In summary, while the logistic function is used to model probabilities in logistic regression, the logit function is used to transform these probabilities into a linear relationship with predictor variables, facilitating the estimation of the regression coefficients.

Question 4

likelihood

Answer 4

maximize the probability of observing of data which is called lifelihood

Question 5

Fitting Logistic Regression Models in R

Answer 5

glm(formula, family, data)

Question 6

ROC curves

Answer 6

ROC (Receiver Operating Characteristic) curves are a graphical representation commonly used in binary classification tasks to evaluate the performance of a classifier.

Question 7

By moving the threshold we’re trading off sensitivity and specificity.

Answer 7

Sensitivity: P(predicted case|true case)
Specificity: P(predicted not a case|truly not a case)

Lowering the threshold means we predict more cases (more y =1 ) increasing sensitivity, but decreasing specificity.

Increasing the threshold means we predict more non-cases (more
y =0), increasing specificity, but decreasing sensitivity.