6+7 Flashcards
What is regression? How does this relate to classification?
Regression involves modelling a relationship between input variables and continuous output variables. Regression is more about approximating functions then classification, which relates to decision making.
What is residual error?
Residual error is the deviation of the output given by the hypothesis from the “true” value,
What is mean squared error?
The average value of the squared error, this gives a measure of the deviation of the hypothesis output from the “true” output.
What is a cost function? What about a cost surface?
A cost function is designed to minimise the cost(MSE) of a hypothesis. A cost surface is a function which describes a surface in 3 dimensions, for each possible w1 and w2 there is a value j
What is parameter search?
Find the minimum point of the cost surface, this is done using optimisation techniques. It involves making an initial guess of the parameter values can computing the cost, then update parameters and compute new cost, repeating the update and compute step.
What is random walk?
Pick the weight update at random, keep the new parameters if the cost gets reduced. Otherwise discard and pick another random change set.
What is steepest gradient* descent?
Weight update corresponds to negative gradient of cost function, this gives the direction that should be taken. This gradient tells how steep a point is, but not how far to travel.
What is a polynomial hypothesis?
One with an input vector which can be to the power of something. The derivative of these can be easily taken to use for steepest gradient
What is a linear hypothesis?
One in which nothing is to the power of anything besides 1. The cost surface of this will be convex and the steepest gradient will be guaranteed to find the optimum solution.
What is a nonlinear hypothesis?
One which is not linear, this will have a non convex cost surface. Steepest gradient is also not guaranteed to find the optimum solution as there may be many local minima.
