Liner regression

Question 1

what is simple linear regression used for

Answer 1

Supervised learning;

quantitative response 𝑌 (dependant) on the basis of a single predictor variable (independent) 𝑋

Question 2

If 𝑓 is to be approximated by a linear function, then it becomes:
𝑌=𝛽0+𝛽1𝑋+𝜖

What does B0 mean

Answer 2

B0 is intercept term: the expected value of 𝑌 when 𝑋 = 0

Question 3

If 𝑓 is to be approximated by a linear function, then it becomes:
𝑌=𝛽0+𝛽1𝑋+𝜖
What does B1 mean

Answer 3

B1 is the slope

Question 4

what is the process of linear regression

Answer 4

1. assess the significance of the coefficients
2. quantify the extent to which the model fits the data

(line of best fit using r squared

Question 5

how is the quality of linear regression assessed

Answer 5

using residual standard error.

EG if RSE = 3.26: actual sales in each market deviate from the true regression line by
approximately 3,260 units on average

Question 6

What happens when more variable are added to a linear regression model

Answer 6

R2 will increase

Question 7

What are the uncertainties when predicting using a MULTIPLE linear regression model

Answer 7

Reducible error: coefficients are only estimates for the true population regression plane

Model bias: linear model (or any other models) for 𝑓(𝑋) is almost always an approximation of reality.

Irreducible error: the response cannot be predicted perfectly because of the
random error 𝜖 of the model

Question 8

Assumptions of the linear model

Answer 8

Additivity: the effect of changes in a predictor 𝑋 on the response 𝑌 is independent of the values of the other predictors. (no other factors impact)

Linearity: change in the response 𝑌 due to a one-unit change in 𝑋 is constant,
regardless of the value of 𝑋 . 𝑗

Question 9

When is linear regression not applicable

Answer 9

1. to order the outcomes eg 1= stroke
2. if the probability is outside 0-1

Question 10

what is logistic regression

Answer 10

Logistic regression estimates the probability of an event occurring, such as voted or didn’t vote (discreet outcome, based on a given dataset of independent variables

Question 11

what makes logistic regression different to linear

Answer 11

It is used to make a prediction about a categorical variable instead of a continuous one.

also has a probability between 0-1

logs are categorical

Question 12

what is the negative of logistic regression

Answer 12

it needs a large data set to have sufficient statistical power to detect a significant effect

Question 13

what is the dummy variable approach

Answer 13

qualitative predictors with the logistic regression mode.

Dummy variables assign the numbers ‘0’ and ‘1’ to indicate membership in any mutually exclusive and exhaustive category

it creates a value of 0 and 1

Question 14

What is linear discriminant analysis

Answer 14

In LDA, we model the distribution of the predictors 𝑋 separately in each of the response classes (i.e. given 𝑌), and then use Bayes’ theorem to flip these around into estimates forPr 𝑌 = 𝑘 𝑋 = 𝑥 .

Question 15

3 reasons to use LDA

Answer 15

When classes are well-separated: parameter estimates for the logistic are surprisingly unstable.

If 𝑛 (data set) is small and the distribution of the predictors 𝑋 is normal
LDA is popular when we have more than two response classes.

it maximises separability

Question 16

what happens if alpha is too small

Answer 16

the optimiser will take a long time to find the minimum

Question 17

what is exploding gradient.

Answer 17

the the slope is vertical so the system will become completely unstable

Question 18

what causes an exploding gradient.

Answer 18

when we have complex models with many para meters and large nural network

Question 19

what is a nested function

Answer 19

functions that embeds another function. as a result of neural link.

Question 20

what is an activation function

Answer 20

function that decides whether information goes from one layer to another.

an example is a step function.

Question 21

what is the difference between bagging and boosting

go over this one.

Answer 21

bagging- multiple models with the same training set

boosting- selecting data points which give wrong predictions.

Each time the data gives a wrong prediction it trains the new model

Question 22

explain the tradeoff between accuracy and interpretability

Answer 22

increasing training data sets may make result more accurate but less easy to digest.

Question 23

between random bagging, boosting and random foresting has the most chance of over fitting when adding more data

Answer 23

boosting because you increase the likelyhood to overtrain the model and the model becomes less effective at predicting future data

Question 24

what is a recommender system

Answer 24

A recommendation system is an AI algorithm, that uses Big Data to suggest or recommend additional products to consumers.

past purchases, search history, demographic information

Question 25

what does machine learning do

Answer 25

Finds a mathematical formula when applied to a collection of inputs (« training data ») produces the desire outputs.

Question 26

what is machine learning

Answer 26

imput+ desired result computation program

Question 26

what is traditional programming

Answer 26

input+ Programm computation = results

Question 27

what are the different types of unsupervised learning

Answer 27

dimension reduction and clustering.

Question 28

what is dimension reduction

Answer 28

a technique used to reduce the number of features in a dataset while retaining as much of the important information as possible.

Question 29

supervised learning

Answer 29

the training set gives the computer example answers. eg pictures of cats and dogs are already provided

Question 30

what does x mean in an algorithm

Answer 30

Input, also known as features or exogenous variables

Question 31

what does Y mean in an algorithm

Answer 31

Output, also known as label, response or endogenous variable: y

Question 32

what does x1 or y1 mean in an algorithm

Answer 32

to collect historical data from a previous algorithm.

Question 33

what are the two types of supervised learning

Answer 33

classification and regression

Question 34

what is classification (what type of learning is it)

Answer 34

Supervised learning we are trying to predict results which have discrete output (i.e. category or class) eg identifying objects or language

Question 35

types of classification

Answer 35

Logistic Regression Linear (and quadratic) Discriminant Analysis K-Nearest Neighbors RLab: Logistic, LDA, QDA, KNN

Question 36

what is regression

Answer 36

we are trying to predict results which have continuous output. like finding the line of best fit stock prices forecast, correlation analysis, medical diagnosis, demand and sales volume analysis,...

Question 37

semi supervised learning

Answer 37

It uses a small amount of labeled data and a large amount of unlabeled data, which provides the benefits of both UL and SL while avoiding the challenges of finding a large amount of labeled data.

Question 38

how does reinforcement learning work

Answer 38

The goal is to learn a policy which is a function (similar to the model in SL) that takes the feature vector of a state as input and outputs an optimal action to execute in that state. The action is optimal if it maximizes the expected average reward. the policy is constantly being updated

Question 39

what is model-based and what is model-free reinforcement learning?

Answer 39

Model-based means memorizing lots of information Model-free means generalize situation eg he self-driving car doesn't memorize every movement but tries to generalize situations and act rationally while obtaining a maximum reward.

Question 40

what is a scalar

Answer 40

a numerical value like 2

Question 41

what is a vector

Answer 41

is an ordered list of scalar values, called attributes, like 𝑎 = −2, 5 .

Question 42

what is a matrix

Answer 42

matrix is a rectangular array of numbers arranges in rows and columns. 2 6 −1 30 −6 −3

Question 43

what is a function

Answer 43

A function is a relation that associates each element 𝑥 of a set 𝒳 (the domain of the function) to a single element 𝑦 of another set 𝒴 (the codomain of the function).

Question 44

where is the local minimum found

Answer 44

We say that 𝑓(𝑥) has a local minimum at 𝑥=𝑐 if 𝑓(𝑥)≥𝑓(𝑐) for every 𝑥 in some open interval 𝑥 = 𝑐.

Question 45

what is a derivative

Answer 45

a function 𝑓 is a function or a value that describes how fast 𝑓 grows (or decreases).

Question 46

what is differentiation

Answer 46

Differentiation is the process of finding a derivative.

Question 47

what is a discreet random variable

Answer 47

a random variable from a distinct data set like a dice can only be random between 1-6

Question 48

what is a continuous random variable

Answer 48

a random variable from an infinite data set

Question 49

what is bayes rule

Answer 49

Conditional probability = the probability of the random variable 𝑌 = 𝑗 given the observed predictor vector 𝑥0 of the random variable 𝑋:Pr (𝑌=𝑗 |𝑋=𝑥0) = Pr 𝑌=𝑗𝑋=𝑥0 Pr(𝑋=𝑥0) ----------------------------------- Pr(𝑌 = 𝑗)

Question 50

what are parameters

Answer 50

variables that define the model learned by the learning algorithm (are directly modified by the algorithm based on the training data).

Question 51

what does it mean if a model has a low biased

Answer 51

the model predicts the training data well

Question 52

what does it mean if a model has high biased

Answer 52

model makes many mistakes on the training data. The line of best fit may underfit the data and may consider the general direction of data.

Question 53

what is the solution of high biased and underfitting

Answer 53

Main reasons: - model is too simple for the data (linear regression) - engineered features are not informative enough Main solutions: - try a more complex model - engineer features with higher predictive power

Question 54

describe a model with low variance

Answer 54

Low variance = low sensitivity = performs well on both train and test sets.

Question 55

describe a model with high variance

Answer 55

High variance = high sensitivity = performs well on train but poor on test overfitting

Question 56

what is a training set

Answer 56

BEFORE Analyst feeds the algorithm input data, which corresponds to an expected output. The model evaluates the data repeatedly to learn more about the data’s behavior and then adjusts itself to serve its intended purpose.

Question 57

what is a test set

Answer 57

AFTER the model is built, testing data once again validates that it can make accurate predictions. Test data provides a final, real-world check of an unseen dataset to confirm that the ML algorithm was trained effectively.

Question 58

what are the causes and solution for high variance (overfitting)

Answer 58

Problems model is too complex for the data (deep NN) -too many features but a small number of training examples Solutions -try simpler model - add more training data if possible - regularize the model (more widely used)

Question 59

what is bagging (bootstrap aggregate) and why is it useful

Answer 59

multiple models of the same algorithm with different random training samples it avoids overfitting data

Question 60

what is boosting

Answer 60

selecting data points which give wrong predictions. Each time the data gives a wrong prediction it trains the new model Often causes overfitting

Question 61

what are the 4 types of classification methods

Answer 61

logistic Linear discriminant analysis (maximises distance) QDA K's nearest neighbours

Question 62

if the boundaries are linear what model of classification shall I use

Answer 62

Linear discriminant analysis and logistic regression

Question 63

How to tell is a model is multivariate

Answer 63

if it has more than one x

Question 64

how to tell if a model is linear and additive

Answer 64

the X is ^to the power linear equation will always be in the form of $y = mx + b$

Question 65

what does r1,r2 mean in an algorithm

Answer 65

they are different leaves on a decision tree

Question 66

what happens is alpha is too large on gradient descent

Answer 66

your optimizer will be jumping big leaps and never find the minimum

Question 67

what happens is alpha is too small on gradient descent

Answer 67

it will take forever to find the minimum

Question 68

what is a perceptron

Answer 68

A perceptron takes several binary inputs 𝑥1 , 𝑥2 , 𝑥3 ,... and produces a single binary output as follows: