Classifier Models Flashcards
(23 cards)
What are classification models?
Classification models are used to classify observations by some discrete label (i.e. gender, revenue bounds)
How do classification models create predictions?
Separating observations using a plane or cluster in order to group them by label
Is classification an unsupervised or supervised learning task?
Supervised
Logistic regression is useful when we have [continuous/discrete/binary] observations and [continuous/discrete/binary] labels.
Continuous, binary
Logistic regression graphs continuous observations onto a graph with only […] label values.
Two/binary
By adjusting the parameters of a logistic regression, we adjust…
The steepness of the ‘middle curve’ in our sigmoid-like function
The accuracy of a classifier is calculated as…
The number of true positives and negatives divided by the total number of predictions
How do perceptrons classify data?
By drawing a hyperplane between the binary data, separating them
What is a superplane?
A hyperplane is a mathematical line that attempts to separate two classes of data
How do perceptrons learn how to properly classify data?
By taking input signals and passing them through a layer that applies a weight to each feature of the input, finishing with a function that converts that output into a binary value like softmax
What does a perceptron’s training phase look like?
Initialise each weight at 0, and cycle through the data. For each x, try classifying it, then update each weight according to some update rule ONLY if it is correct
What is the problem with a basic perceptron?
They have linear separability, meaning they can only separate the data with one line, making more complex shapes difficult
What is a multilayer perceptron?
An adaptation of a basic perceptron that uses multiple weighted layers to simulate multiple layers of neurons, called hidden layers
What are the layers between the input and output of a multilayer perceptron called?
Hidden layers
What is K-Nearest Neighbours, and what type of problem is it used for?
A supervised learning algorithm that models similarity via distance, and it is used for classification problems
How does K-Nearest Neighbours facilitate predictions?
Splitting the dataset into clusters, each of which represent a different class cluster, predicting new values as the cluster it is closest to
How does K-Nearest Neighbours make a prediction given one new data point?
Calculate the distance to the other points, then sort that list and select the K nearest points. Find the majority class among those 3 neighbours to find the new point’s class
How can outliers affect K-Nearest Neighbours?
Since we predict using a distance metric, outliers can skew how often we predict a certain class given its position on the graph
How can class imbalance affect K-Nearest Neighbours?
Our predictions are based on a number of nearest neighbours, so if we simply have more neighbours than the other class, we are more likely to select the other, even if the nearest one is more accurate
How can we select the optimal parameters for K-Nearest Neighbours?
Starting with one and increasing k by 1 each time, or setting k to the square root of the number of data points in the training dataset
What is Weighted KNN?
A variant of KNN where we make the assumption that the impact of nearer neighbours should be greater than the further neighbours, using distance to consider impact
Why does complexity increase with the size of the training data?
KNN is a form of instance-based learning, meaning they construct hypotheses directly from training instances, therefore as the data gets larger, we encounter slower training - O(n)
KNN is most suited to [higher/lower] dimensional data.
Lower
