5) Convex optimisation

Question 1

What is an objective function, and how are global and local minimisers defined

Answer 1

Question 2

What is a necessary condition for a local minimiser of a continuously differentiable function

Answer 2

Question 3

What are key properties of minimisers for convex functions defined on convex sets

Answer 3

Question 4

For the objective function Φ(w)= 1/2 ∥Aw−b∥ ^2 , what condition characterises global minimisers

Answer 4

Question 5

What is the basic idea of gradient descent for convex optimisation

Answer 5

Question 6

In gradient descent, why is the direction of fastest descent given by the negative gradient

Answer 6

Question 7

What is exact line search in gradient descent

Answer 7

Question 8

What does it mean for a function to be Lipschitz continuous

Answer 8

Question 9

What convergence guarantee does gradient descent have for convex, smooth functions with Lipschitz continuous gradients

Answer 9

Question 10

What is stochastic gradient descent (SGD), and how does it address the computational cost of large datasets

Answer 10

Question 11

What convergence guarantee does stochastic gradient descent (SGD) have for convex, smooth functions

Answer 11