Support Vector Machines (SVMs)

Question 1

What are support vectors?

Answer 1

Training examples that are exactly on the margin

Question 2

What is a consequence of having a short margin γ?

Answer 2

It leads to misclassification

Question 3

Why do we maximise the margin?

Answer 3

To avoid overfitting

Question 4

What is the margin?

Answer 4

The margin, γ, is the perpendicular distance between the decision boundary and the closest training example.

Question 5

What do we assign xo in SVMs?

Answer 5

We don’t assign a value for xo in SVMs

Question 6

What is the formula for the margin?

Answer 6

dist(h, x(n)) = |h(x(n))| / ∥w∥
where ∥w∥ = square root (wTw) is the Euclidean norm

Question 7

What does w0 represent in SVMs?

Answer 7

Bias

Question 8

What is the constraint when calculating the margin?

Answer 8

y(n).h(x(n)) > 0, ∀(x(n), y(n))∈𝒯
(All training examples have to be correctly classified)

Question 9

What is the affect of scaling w and b on the hyperplane?

Answer 9

There is no affect on the position of the hyperplane

Question 10

What changes can we make to the constraint for SVMs?

Answer 10

By dividing w and b by min n y(n) h(X(n)), we can set the constraint equal to 1:
min n y(n).h(x(n)) = 1, ∀(x(n), y(n))∈𝒯

Question 11

Whats the equation for minimising the margin?
(for the constraint min n y(n).h(x(n)) = 1, ∀(x(n), y(n))∈𝒯)

Answer 11

argmax w,b = {1 / ∥w∥}
argmin w,b = {∥w∥}

Question 12

How can we change the constraint so that the optimal solution satisfies for at least one training example?

Answer 12

min n y(n).h(x(n)) >= 1, ∀(x(n), y(n))∈𝒯

This is equivalent to the other constraint as the smallest value for ∥w∥ happens when the constraint is 1.

Question 13

Whats the equation for minimising the margin?
(for the constraint min n y(n).h(x(n)) >= 1, ∀(x(n), y(n))∈𝒯)

Answer 13

argmin w,b = {1/2 . ∥w∥^2}