Supervised Learning Flashcards
(24 cards)
What is a Support Vector Machine (SVM)?
A supervised machine learning model that finds a line that separates data points by some form of margin
What types of problems are Support Vector Machine (SVM) used for?
Classification, regression and clustering
What is a margin in an SVM?
The distance separating the closest pair of data points belonging to opposite classes
How can we train an Support Vector Machine (SVM)?
We want to optimise the model such that the margin is maximised
The two data points that are the closest ones belonging to opposite classes are called the support vectors, as they define the…
Decision boundary
How do we deal with outliers in Support Vector Machines (SVMs)?
By intentionally allowing misclassifications, letting outliers be classified incorrectly
If the training data contains outliers, the maximum margin classifier would be…
Closer to the green observations than from the red
If the data is not linearly separable for a Support Vector Machine (SVM), then we may…
Increase the dimensionality of the data via some transformation to make the classes linearly separable
What is the kernel trick in an Support Vector Machine (SVM)?
We represent the dataset as an nxn kernel matrix of pairwise similarity comparisons, with each entry defined as the dot product of the vectors in the feature space
Why do we need the kernel trick in Support Vector Machines (SVMs)?
Transforming the data to higher dimensions has a high computation cost, therefore the kernel trick lets us calculate the relationships in the data without actually transforming it
What is a decision tree?
A supervised learning algorithm that uses a sequential model of decisions and their possible consequences to produce predictions
How do decision trees produce predictions?
We follow the decision tree’s logic until we reach a leaf, which tells us the value, sort of like a flow chart
What is a classification tree?
A variant of a decision tree that classifies observations into categories based on multiple input values
What is a regression tree?
A variant of a decision tree that is used to predict numeric values - though, this is still, really, a classification, as we can only predict the bounds of that value
How do classification trees produce predictions?
Start at the top, and work your way down until you get to one of the tree’s leaves - typically, if a statement is true, you go left, and if it’s false, you go right
How can we build a classification tree?
Choose which question to ask at the top of the tree, then continue this process for the other columns
What is impurity in a decision?
A measure of how mixed the data is regarding its outcome
What is Gini impurity?
A metric for quantifying impurity as 1 minus the sum of the probability of the outcome squared
How do we adjust Gini impurity for non-categorical data?
Sort the observations in that column, then calculate the average age for all adjacent rows, calculating Gini impurity for each average
What issues can we face when dealing with very small amounts of data, or class imbalance?
Bias or overfitting
What is the pruning of a classification tree?
A data compression technique that reduces the size of a decision tree by removing non-critical or redundant sections, reducing complexity and thereby improving generalised accuracy
What is pre-pruning of a classification tree?
Pruning while training. One method is specifying the minimum number of samples that must be present in each node, and another is maximum depth
What is post-pruning of a classification tree?
Pruning after training. We start at the lowest branch, and consider the error of the whole tree, and the error of the tree minus the lowest branch. Repeat until the error of the whole tree is less than the lowest branch
