Constraint Optimisation - Inequality Constraints

Question 1

What are the two types of constraints and what are their properties?

Answer 1

Inactive constraints: Where the constraint does not actually matter
Active constraints: Where the constraint does actually matter

Question 2

What is the value of λ for inactive constraints?

Answer 2

λ = 0

Question 3

What is the requirements for the gradient of the constraint and the function for an inequality constraint to be active?

Answer 3

They must be parallel and be in the same direction of each other. This means that the direction of acceptable values is in the direction of the anti-gradient.
[Picture 13]

Question 4

What is the requirements for the gradient of the constraint and the function for an inequality constraint to be inactive?

Answer 4

They must be parallel and be in the opposite direction of each other.
[Picture 14]

Question 5

When we pick a single vector to solve for w0, what problem can this present?

Answer 5

If there is noise on the vector we chose, this can be a real problem so instead we take all support vectors and calculate w0 for each of them and then average the result