General Flashcards
Lagrangian multiplier
used to find minimum given a equality constraints
what is newton method
used to find the roots of a polynomial by drawing tangent at a point and seeing where it meets the X axis and then checking the value at that point.
what does min ||Ax-b||^2 will do?
finds a hyperplane by changing the value of A and b for the point’s x of dimension n. such that the least square error is minimum
what is closed and open form solution?
when you have a unique solution to the equation it is called closed form solution.
define dot product?
dot product of two vector is finding the projection of one vector on other. or finding X^t y
what is nth norm of a vector?
it is the nth root of sum of nth power of each directional coefficients of the vector.
first norm is just sum
second norm is called eucledian distance sqrt(X^t X)
inifinty norm is the max
define optimization?
devising the effective decission / prediction.
Define these terms Linear function Affine function linear programming Quadratic programming inequality and equality constraints feasible set
if function maps follows the following F(x+y) =F(x)+F(y) and F(kx)=k*F(x) then the function F is linear ie it maps everything in a line the polynomial it needs is 1 and there is only one minimim. map passes through origin.
Affine is linear + a constant term it is of the form Ax+b it is linear transformation followed by translation it follows only F(x+y) =F(x+ F(y).
linear programming Linear cost function min C^T X with affine inequalitites.
define Hyper plane
Half spaces
it is a plane of one dimension lesser than the original space and it divides the space into two parts, each half is called half spaces.
what does regularization do? and use of different norm for sparsity?
regularization is used to increase sparsity it is done by solving an optimization problem min J(x) s.to Ax=b this J(x) can be a regularization function
* Eucledian norm is one such regulatization function which finds the intersection of circle defined by the norm and the hyperplane defined by the constraint. it brings the size of each elements.
*0 norm finds the number of non zero elements
* for 1 norm the function is rhombus in shape and it promotes sparsity
* lp norm where p is less than 1 also promotes sparsity
but lp norms and l 0 norm are not continuous so we dont use them.
When is a function convex?
Line joining any two points lie above the curve.
Tangent lies below the curve ie f”(x) >= 0
When is a problem considered as convex optimization problem?
When objective function is convex and feasible set is convex set.
What does x <= y (curvy less symbol) means?.
If x and y are vectors then it is element wise comparison. If it is matrix then Y-X is a martix which is poisitive semi definite.
Some operations that preserves convexity?
Intersection.
Scaling.
Translation.
Projection(projecting R3 to R2)
Difference between minimum and minimal?
Unlike number where every point can be compared, matrices and vectors can’t be compared so we need other notion than minimum this is where minimal comes in picture.
In the given set if a point has no point below to orthogonal horizontal and vertical plane that point is called a minimal point.
