Non-linear Transformation Flashcards
(12 cards)
Why do we use non-linear transformations?
Because for non-linearly separable problems, a decision boundary of wTx = 0 will lead to underfitting
What is an example of the basis expansion?
ϕ(x) = (1, x1, x2^1)T
Can we use linear models in this new space?
Yes, because the new transformed space becomes linearly separable
What is a polynomial decision boundary of degree p?
A decision boundary of degree p uses a feature transform that includes all terms of order <= p based on the input X.
What is a feature transform not suitable?
When there are a lot of input variables
What is the decision boundary of the transformed space?
wTϕ(x) = 0
What is the decision boundary of the original space after using a basis expansion?
wTϕ(x) = 0, but replace ϕi(x) with the corresponding
value that depends on x.
What can we use for non-polynomial transformations?
e^x
Should you exclude basis functions that aren’t used in the original decision boundary?
No, we don’t know beforehand what values will be needed.
Why is the logistic regression still linear?
Because it is still linear w.r.t its parameters w even when transforming into a different space with a nonlinear decision boundary in its original space.
Why do we use linear models?
Because they are faster, more robust and have better generalisation.
What is a consequence of high number of dimensions?
It can lead to overfitting
