Classification

Question 1

classification probabilities

Answer 1

P(class|features) we’re essentially trying to understand the likelihood of an observation belonging to a particular class given its features.

Question 2

Approaches to Classifcation

Answer 2

Generative classifiers:.
Generative classifiers try to understand how data is generated, modeling both the features and the classes together.
These generate probabilities P(class|predictors) by first estimating other distributions.
Rely on statistical theory like Bayes theorem.

Discriminative classifiers:
Discriminative classifiers focus on predicting the class directly based on the observed features, without necessarily understanding the underlying data generation process.
Estimate P(class|predictors) directly.
Also referred to as conditional classifiers.

Question 3

prior probability for a class

Answer 3

The “prior probability for a class” refers to the probability of a particular class occurring before considering any evidence or features. It represents our initial belief or assumption about the likelihood of each class before we observe any data.

Question 4

Bayes’ Theorem

Answer 4

Bayes’ Theorem to determine the probability of a class given a set of features (or a feature vector).

Question 5

Posterior Probability for a Class:

Answer 5

P(j∣x) represents the probability of class j given a feature vector x. This is what we want to find out—it’s like the updated probability of each class after we’ve observed the features.

Question 6

misclassification rate

Answer 6

The performance of a classifier is usually measured by its misclassification rate. The misclassification rate is the proportion of observations assigned to the wrong class.

Question 7

Linear Discriminant Analysis

Answer 7

Linear Discriminant Analysis
- Linear Discriminant Analysis is often abbreviated to LDA.
- LDA applicable when all the features are quantifiable.
- We assume that fj is a (joint) normal probability distribution.
- In addition, we assume that the covariance matrix is the same from class to class.
- The classes are differentiated by locations of their means.

Question 8

Kernel Discriminant Analysis (KDA):

Answer 8

KDA extends LDA by allowing for more complex, nonlinear decision boundaries between classes. It achieves this by mapping the data into a higher-dimensional space using a kernel function. KDA essentially “lifts” the data into a higher-dimensional space where the classes might be more easily separable and then applies LDA in this new space.