Session 9- Logistic Regression Flashcards
What is the basic principle of penalised methods?
Improve prediction accuracy by reducing the variability of the regression estimates at the cost of increased bias
Penalised/ Regularized Regression allows what?
Automatic variable selection by shrinkage:
The coefficients of the weaker predictors are shrunk towards zero.
- useful for high dimensional data if the number of variables is close or larger than sample size
- deals effectively with multi-collinearity
Variance bias trade off due to shrinkage usually results what?
A model which predicts better unseen case than normal regression
What is a big advantage of regularized regression methods especially if using lasso or elastic net?
They also perform automatic variable selection
What is
popular if have small number of variables but some of them are highly correlated to get a more stable result/model?
Ridge
What can be easily extended from linear regression for continuous outcomes to a larger class of models, including:
Logistic, Multinomial, Ordinal, Poisson and Cox regressions?
Regularized Logistic Regression
Regularized Logistic Regression is usually referred to as what?
Generalized Linear Models
What does Logistic regression allow modelling of?
Relationship between binary outcomes, such as recurrence of psychoses (yes/no), as a non-linear function of our predictor variables.
Other models can be fitted in a similar way, for example time to event data using Cox-regression
What are problems with linear regression?
Does not make sense for one binary outcomes
We rather assume a nonlinear relationship
Output variable is limited to [0,1], but with linear regression some of our observations are outside this range
Our goal is to separate best the two groups not to minimize MSE. Linear regression would be highly sensitive to influential cases
Assumptions of linear regression are violated (esp. homogeneity of variances) and hence inference is not valid
What is an s-shaped/ sigmoid relationship?
A non-linear relationship
How can a sigmoid relationship be modelled by the logistic function?
f(x)= 𝒆𝒂+𝒙)/(𝒆𝒂+𝒙+𝟏)
How is Logistic function flexible?
If vary alpha will have different logistic functions
If vary x can also obtain very different functions - If change x/Beta e.g., our age relationship can vary
The simple linear regression describes the linear relationship between the outcome y and the predictor variable x.
𝜖 describes the random component (error) and is assumed to be normal distributed.
In a logistic regression we….
Relate x and y by way of a function, known as link function g():
g(𝑦)=𝛽0+𝛽1 𝑥1+𝜖
so that we can model a linear relationship between the left and right hand side of the equation.
We use the so-called logistic function to produce our models output:
P(Y=1|X)= 𝑒𝛽0+𝛽1 𝑥1 /𝑒𝛽0+𝛽1 𝑥1 +1)
Here, we are modelling the probability that our outcome belongs to class 1 given the input feature variable 𝑋1.
- What is the underlying probability distribution of this logistic model?
- How can we rearrange this function? and what is the term on the left side known as?
- The Bernoulli distribution.
- 𝑙𝑛⌊(𝑃(𝑌=1|𝑋)/1−(𝑃(𝑌=1|𝑋)⌋=𝛽0+𝛽1 𝑥1
Term on the left side logit-link function!
We use the so-called logistic function to produce our models output:
P(Y=1|X)= 𝑒𝛽0+𝛽1 𝑥1 /𝑒𝛽0+𝛽1 𝑥1 +1)
Here, we are modelling the probability that our outcome belongs to class 1 given the input feature variable 𝑋1.
- What is the underlying probability distribution of this logistic model?
- How can we rearrange this function? and what is the term on the left side known as?
- The Bernoulli distribution.
- 𝑙𝑛⌊(𝑃(𝑌=1|𝑋)/1−(𝑃(𝑌=1|𝑋)⌋=𝛽0+𝛽1 𝑥1
Term on the left side logit-link function!
