Q

Question 1

How do you derivate a0 and a1 for a linear regression model that is least squares optimal?

Answer 1

1.Identify the sum of squared errors equation for the model
2.Take partial derivatives with respect to a0 and a1
3. Set equal to zero, and solve simultaneous equations

Question 2

What does the R^2 statistic indicate?

Answer 2

If it is close to 1, the model is able to capture variation in the data well

Question 3

What is meant by regularization?

Answer 3

Regularization is the application of constraints to the amplitude of estimated model parameters in order to simplify the model

Question 4

What is regularizations relevance to modelling?

Answer 4

it reduces the model variation and helps with predictor selection
also reduced numerical issues when constructing the regressor matrices

Question 5

What does LASSO stand for?

Answer 5

Least Absolute Shrinkage and Selection Operator

Question 6

What does Lasso do to the cost function J?

Answer 6

Introduces a term into cost function J which is linear with absolute values of model coefficient, through tuning parameter Lambda

Question 7

What are the parameters in regularization governed by?

Answer 7

the set of parameters is identified by the point at which contours of J and the additional parameter-dependent contours are tangential

Question 8

What are the steps for applying Lasso through cross-validation?

Answer 8

1 - Divide n observations into K equal groups
2 - Specify a range of Lasso weights lambda (lambda 1… lambda m)
3 - let k=k+1 and lambda =lambda k, train K different models using each of the data groups and lasso, except 1
4 - Find average performance
5 - if K<M return to 3, otherwise find k* such that CVMSE is minimum
6 - Let lambda=lambda k* and apply lasso to n observations to determine final model parameter estimates.

Question 9

What is the advantage of Lasso over L2 regularization?

Answer 9

-optimal set of parameters after regularization is tangent point of J cost contour with constraint contours
- The Lasso formulation makes this point null -> induces sparse solutions
- Some predictors can be excluded, reducing model size.

Question 10

What is the optimal set of parameters after regularization?

Answer 10

the tangent point of J cost contour with constraint contours

Question 11

What is meant by Odds?

Answer 11

The ratio between the probability of an event occurring with the probability the event won’t occur

Question 12

What is the definition of logistic model?

Answer 12

The association of posterior probability y of the class membership to a logistic sigmoid function.

Question 13

What is a support vector machine and how does it work?

Answer 13

-SVM is an extension of the maximal margin classifier
- classifier maximises the margins of support vectors
- SVM expands feature space through non-linear features and returns classification of the original data set.

Question 14

How does an SVM work?

Answer 14

• works by defining a separation hyperplane in the data space through a small number of data points that are closest to the hyperplane(support vectors), and maximising them

Question 15

What is the main disadvantage of a support vector classifier?

Answer 15

• Hard classifier - separated classes without returning any info on the level of confidence for each data point.

Question 16

What is meant by ARX?

Answer 16

Auto Regressive model with Exogeneous inputs

Question 17

What are ARX and ARMAX model?

Answer 17

linear models used in the analysis of time series data modelling of dynamical systems through a sample of its present and past inputs/outputs.

Question 18

What is mean by ARMAX?

Answer 18

Auto Regressive model with Moving Average Exogeneous inputs.

Question 19

What is a confusion matrix?

Answer 19

table to compare the rate of the true and predicted values of a classifier model

Question 20

What is mean by ROC and how is it constructed?

Answer 20

• Receiver Operating Characteristic
• A diagram formed by plotting the sensitivity against 1-specificity, when the classifier threshold value changes from T=0 to T=1.
• each point on the curve defines the value of TPR and FPR obtained for a certain threshold of the classifier
  -can assess accuracy over all thresholds

Question 21

What is the AUC and what is it used for?

Answer 21

Area under Curve
quantifies performance of a classifier
perfect classifier has AUC of 1

Question 22

What elements of the data should be considered before building a black box model?

Answer 22

• Model will represent the data using basis functions
• Provides the best representation, but the data itself must be of good quality
• Density of the data shapes the relative weights of the basis function components
• Low density areas will be poorly represented and overfit

Question 23

How does cross validation work simply?

Answer 23

Rotate training and testing samples through the data
test on one part of the data and train on the rest

Question 24

What are the two most important factors to employ to prevent bias in AI models?

Answer 24

Pre-processing of data
Explainability of method

Question 25

What does PCA application do to a data set?

Answer 25

PCA projects and reconstructs feature set into a frame set that maximises variability in each axis

Question 26

What is the purpose of PCA?

Answer 26

Provides a measure of importance of each input and allows the reduction of the dimensionality of a large feature space

Question 27

What is the geometrical relationship between principal components?

Answer 27

Each PC is orthogonal to the next

Question 28

What do the eigenvectors of a data set represent?

Answer 28

Principle components

Question 29

What do the eigenvalues of a data set represent?

Answer 29

Variance contained in that eigenvector direction

Question 30

How do you identify the first principle component of a data set?

Answer 30

The largest eigenvalue points to the eigenvector containing the first PC

Question 31

How would you incorporate non-linearity into the decision boundary of a two-input logistic classifier?

Answer 31

Rewrite the basis function and change the number of weights required.

Question 32

what is meant by inner product?

Answer 32

dot product

Question 33

How do you derive the least squares solution?

Answer 33

(z - Xa)T (z - Xa) and expand this like a standard equation
gradient with respect to m-vector
nabla (ZTZ - 2aT XT z + aT XT Xa)
= 0 - 2 XT a + 2 XT Xa
equate to zero and rearrange
a = (XTX)^-1 XTz

Question 34

You introduce quadratic non-linearity to a model, what are the new column dimensions of the design matrix?

Answer 34

(d+n)! / d! n!, where n=2 and d=3