L18 - Support Vector Machine

Question 1

What type of model is a SVM?

Answer 1

Non-probabilistic Supervised Classification.

Question 2

What is the goal of SVM?

Answer 2

Find the optimal plane through a set of data points. That being, the one with the largest margin from the plane to the support vectors.

This enables greater classification capability.

Question 3

In a feature space, what do the dimensions represent?

Answer 3

Features. I.e if there are 5 features, there are 5 dimensions in the feature space.

Question 4

How does SVM classify data?

Answer 4

The feature space is partitioned, each partition represents a class. The SVM places the data about the partition planes to classify it.

For example, 2 dimensions, data would be place above or below the plane for classification.

Question 5

What does it mean the SVM is non-probabilistic?

Answer 5

The classification is based purely on features of the data point.

Question 6

What does the Kernel Trick enable SVM to do?

Answer 6

Make classification of non-linear data

Question 7

What are Support Vectors?

Answer 7

Data points closest to the separation plane.

They are called this since the support the classification decision boundaries.

Question 8

Define the Maximum Margin Classifier…

Answer 8

When we classify based on the threshold that gives us the largest margin between the place and the support vectors.

Question 9

What is an issue with using Max. Margin Classifier?

Answer 9

Prone to outliers.

Question 10

What is the solution to the outlier problem?

Answer 10

Enable the SVM to make misclassifications.

Question 11

What is the Soft Margin of the SVM?

Answer 11

If misclassifications are allowed, the soft margin is the margin between the support vectors and the separation plane.

But the soft margin enables misclassifications, meaning that outliers on the wrong side of the separation plane will be classified to prevent outlier impacting classification.

Thus, misclassification is allowed to enhance the general classification capability.

Question 12

How do we know if we have non-linear classification?

Answer 12

If data overlaps in the features space.

Question 13

If we have a non-linear problem, what transformation do we perform on the data?

Answer 13

A mapping transformation in which overlapping features are bought into higher dimensions. This means if the data is not linearly separable in one dimension, it should be separable in a higher dimensional space.

Question 14

What is the name of the function that performs higher dimensionality mapping? Given an example of one

Answer 14

Kernel Function

Example : X mod 2

Question 15

What is the issue with non-linear SVM?

Answer 15

Computational cost of operations increase greatly when they have to be performed in higher dimensions.

This is infeasible for large data.

Question 16

What is the solution to the high computational cost of kernel functions in high dimensional space?

Answer 16

Kernel Trick - Enables the data transformation to be applies without moving to higher dimensional feature space.

Question 17

How does the Kernel Trick work?

Answer 17

Create a Kernel Matrix M = NxN where N is the number of data points in the feature space.

Kernel Function performs a dot product on each matrix cell M_i,j

Question 18

What is a Kernel Function?

Answer 18

A function that takes 2 feature vectors from the features space, and returns the dot-product to the feature space.

Question 19

What type of value does a Kernel Function return?

Answer 19

A real number.

Question 20

What are some Advantages and Disadvantages of SVM?

Answer 20

Advantages:
- Effective in higher dimensions
- Capability of non-linear classifications
- Memory efficient

Disadvantages:
- If feature count is greater than the number of data points, the feature space will be greater than the sample count. This will lead to poor classification capability.