CHAP 6 : ML Linear Regression Flashcards
What is linear regression?
- It is a supervised machine learning algorithm
- that finds the best fit linear line between independent and dependent variable (i.e. finds linear relationship between independent and dependent variable)
There is only one optimal line that best fits data in linear regression. Is the statement true or false?
True
How to calculate error function?
For N number of data points,
Take the sum of squared error between y value of each data point and y hat (predicted y value) on regression line multiply by 1/2N
- Error function is the average sum of squared errors
For error function, why do we need to take the average squared error sum (i.e. why do we need to divide by N)
It is to remove the dependency on the data set size. The error magnitude is thereby independent of N.
For error function, sine it is average squared error sum why do we need to divide by 2 in the denominator –> divide by 2N instead of just N
It is to make calculations neater when finding the derivative, the cancel out the 2 in the numerator
How to calculate new value of W0 and W1 (parameters)?
W0’ = w0 - L(1/N) (partial derivative) ; W1’ = w0 - L (1/N)(partial derivative) , where L is the learning rate
What is the learning rate, L?
It is the rate that controls the rate of change of values of W0, W1 at each step
What happens if the learning rate is too large?
The algorithm may not converge (and sum of squared error may increase)
What happens if the learning rate is too small?
The gradient descent function will take a very long time to converge
What are the steps in gradient descent function? [4]
- Start with a random value of (W0, W1) and define learning rate (e.g. L = 0.000001)
- Calulate partial derivative of error function with respect to W0 and W1.
- Update current values of W0, W1
W0’ = w0 - L (partial derivative) ; W1’ = w0 - L (partial derivative)
- Repeat steps 2-3 till convergence
How does the gradient descent function know when to stop?
It stops when it converges.
How can we define the convergence of the gradient descent function?
We can set a convergence condition to be achieved OR run the algorithm for a fixed number of iterations
**Argmin, by definition, tries to find the lowest cost value of the objective/error function. Is the statement true or false?
False. It tries to find the INPUT VALUES that minimises the cost of the error function
Gradient descent algorithm is used to find the least optimal parameters and eliminate them. Is the statement true or false?
False. It is used to find the most optimal parameters.
When learning rate is large, then the gradient descent algorithm takes a long time to converge because the weights are updated with smaller values. Is the statement true or false?
False. When the learning rate is large, the gradient descent algorithm may overshoot the optimal solution and oscillate around it, or it may even diverge and fail to converge.
This is because larger learning rates cause larger updates to the weights in each iteration
Order the steps for the gradient descent algorithm as shown below:
A) Updating of w0 and w1 values
B) Iteration of the previous two steps till convergence or termination
C) Setting a random w0 and w1, and a learning rate
D) Finding the partial derivates of w0 and w1
C, D, A, B
There is no optimal way to attain a good L value (i.e., it requires trial and error to choose a nominal value of L). Is the statement true or false? Note: L is your learning rate.
True
What is the overall flow for linear regression ? [6 steps]
1) Get Dataset,
2) Plotting of dataset,
3) Error function,
4) Gradient Descent algorithm,
5) Creation of Linear Regression model and
6) Prediction.
It is possible to use linear regression when there is no dataset. Is the statement true or false?
False. Linear regression is a supervised learning algorithm that requires a labeled dataset, i.e., a dataset that contains both the input features and the corresponding target variable values.
The key difference between single variable and multivariable linear regression is the number of independent variables. Is the statement true or false?
True
Using linear regression, there are two known values of x and y, namely (3,5) and (8,10). What are the values of W0 and W1?
W0 : 2
W1 : 1
Using linear regression, there are two known values of x and y, namely (3,5) and (8,10). What are the values of W0 and W1?
W0 : 2
W1 : 1
Given that w0,w1=[2,10], find the value of y when x=3.
32
Given that w0,w1,w2,w3= [8,10,-2,4], find for y when x1=3, x2=5, x3=10.
68
