Exam prep

Question 1

Supervised learning

Answer 1

A subcategory of machine learning which uses inputs, a desired output and labels to train a model.

Question 2

Semi-supervised learning

Answer 2

Using labelled as well as unlabeled data to perform certain learning tasks.

Question 3

Dimensionality reduction

Answer 3

Model h in H that represents each instance x with a lower dimension feature vector whilst preserving key features.

Question 4

Unsupervised learning

Answer 4

A subcategory of machine learning which uses inputs and a desired output to train a model.

Question 5

Clustering Analysis

Answer 5

A machine learning technique that involves grouping sets of objects in such a way that objects in the same group have similar features.

Question 6

Anomaly Detection

Answer 6

The identification of rare items, events or observations which deviate significantly from the majority of the data.

Question 7

Reinforcement learning

Answer 7

A subset of machine learning that allows an agent to learn through trial and error using feedback from its actions.

Question 8

P(A | B)

Answer 8

P(A n B)/P(B)

Question 9

Max

Answer 9

Maximum value of a function

Question 10

Argmax

Answer 10

Sequence of x values to get maximum y values.
For example, for max sin(), values would include 0.5, 1.5 etc, which maximise the y value, but would not include things like 1 or 2, which do not maximise sin().

Question 11

MAP(A)

Answer 11

argmax_a P(A)
Highest probability in A.

Question 12

MAP(A, B)

Answer 12

argmax_a, b P(A,B) = argmax_a, b P(B|A)P(A)
Highest probability in P(A, B)

Question 13

Validation dataset

Answer 13

Part of training dataset.

Question 14

Regularisation

Answer 14

Techniques that are used to calibrate machine learning models in order to minimize the adjusted loss function and prevent overfitting or underfitting.

Question 15

Test dataset

Question 16

Training dataset

Question 17

Error rate for confusion matrix

Answer 17

1 - accuracy
or
FP + FN / TP + FP + FN + TN

Question 18

Accuracy for confusion matrix

Answer 18

TP + TN / TP + FP + FN + TN

Question 19

Loss function

Answer 19

1/m x sum(x - x_)^2

where
x = observed values
x_ = predicted values

Question 20

Linear regression

Answer 20

Used to minimise the loss function.

Question 21

Naive Bayes

Answer 21

P(X … Xn | Y) = ΠP(Xi | Y)
Essentially, Xi and Xj are conditionally independent given Y.

Question 22

Rosenblatt’s Perceptron

Answer 22

• initalise weights randomly
• take one sample x and predict y
• for erroneous predictions update weights
• if output = 0 and y’s true value 1, increase weights
• if output = 1 and y’s true value 0, decrease weights
• repeat until no errors

Question 23

Perceptron

Answer 23

Machines for binary classifications, with one weight per input.
We multiply the weights with their respective inputs and add bias.
If the result is larger than threshold return 1, else 0.
XOR requires multiple perceptrons.

Question 24

Convolution filter

Answer 24

Apply filter to input via every stride, and calculate results by multiplying matching cells and adding together for that space the filter is on.

Question 25

Output size for convolution filter

Answer 25

(N-F) / stride-1
where
N = length or width of input
F = length or width of filter

Question 26

Stride

Answer 26

Amount the filter jumps from each input.

Question 27

Manifolds

Answer 27

A manifold is a mathematical object that can be curved but looks flat locally.

Question 28

V

Answer 28

Or in decision trees.

Question 29

Ʌ

Answer 29

And in decision trees.

Question 30

Regression for nearest neighbour

Answer 30

Summation of classifiers of k nearest neighbours / k

Question 31

Pooling

Answer 31

Reduces the matrix by only choosing important features.

Question 32

Max-pooling

Answer 32

An extension of pooling which we pick the maximum of a selected group of numbers to represent the bigger matrix as a smaller one.

Question 33

Indirect Casual Effect

Answer 33

For X -> Y -> Z
Iif Y is observed, X and Z are independent and cannot affect each other.

Question 34

Indirect Evidential Effect

Answer 34

For Z -> Y -> X
Iif Y is not observed, then X can influence Z.

Question 35

Common Cause

Answer 35

X <- Y -> Z
If y is not observed, then X and Z can influence each other.

Question 36

Common Effect

Answer 36

X -> Y <- Z
If Y is not observed, X and Z cannot influence each other.

Question 37

Causal Reasoning

Answer 37

Cause to effect, where we have a certain cause which predicts an effect.

Question 38

Evidential Reasoning

Answer 38

Effect to cause.

Question 39

Intercausal reasoning

Answer 39

Based upon influential chains, aka two nodes which are not directly connected.
An example is explaining away.

Question 40

Explaining away

Answer 40

The probability of a cause given its effect is lower when an alternative cause is present and higher when the alternative is absent.

Question 41

Zero-padding

Answer 41

Surrounding the matrix with cells containing 0 to prevent shrinkage of data.

Question 42

ReLU Layer

Answer 42

Determines numbers below 0 and converts them into 0.

Question 43

Residual Block

Answer 43

Adds original input to the output.

Question 44

Dropout

Answer 44

Removal of certain nodes.

Question 45

BatchNorm Layer

Answer 45

Uses batch normalisation to make trainings of neural networks faster and more stable.

Question 46

Softmax Layer

Answer 46

Adds decimal probabilities to each class in a multi-class problem.

Question 47

Preprocessing Data

Answer 47

Ensures that data is in a format that the network can accept, which includes re-sizing and re-scaling into an appropriate scale.

Question 48

Accuracy

Answer 48

|{M(X) = lx | x exists in D}| / |D|
where
M(x) - predicted label
lx - true label

Question 49

Empirical Loss

Answer 49

Measures the loss on a smaller scale.

Question 50

Generalisation Error

Answer 50

The measure of how accurately an algorithm can predict outcomes based on unseen data.

Question 51

For neural network gradients

Answer 51

1) Substitute data in (where we have y1, substitute what y1 actually is)
2) Simplify the expression down
3) Find the derivative (x1, x2, x3 etc) and the number associated with the question, and find if the derivative matches the coefficient in the equation.

Question 52

For InfoGains

Answer 52

1)
Workout the original entropy of y.
2)
Workout the conditional entropy, calculating all possibilities and centralising X.
3)
subtract the conditional entropy from the original entropy.

Question 53

For k-trees (regression)

Answer 53

1)
Figure out the metric that is wanted and workout distances from that metric.
2)
Figure out the k closest neighbours from the coordinates you want to plant.
3)
Figure out the classification of those neighbours and perform the following equation:
Summation of classifiers of k nearest neighbours / k

Question 54

Forward Computations

Answer 54

• collect annotated data.
• define a model and initalise randomly
• predict based on current model (forward propagation)
• evaluate predictions

Question 55

Backward Computations

Answer 55

• compute loss
• compute loss gradient over weights
• update weights
• completion of a round of forward-backward computation.

Question 56

Cross validation v random resampling

Answer 56

Random resampling randomizes training set and test set after every iteration, potentially leading to similar test and training data. Cross validation prevents this by selectively choosing test data every iteration, specifically by the index.

Question 57

I-map

Answer 57

Represents independencies with graphs.
Depending if two nodes are dependent or not, and what the table of data represents, the graph can be an I-map of the table.
So if the graph showed independence between two nodes and the table shows this too, then we can class the graph as an I-map of the table

Question 58

I-map exceptions

Answer 58

Following on, this implies that if a graph shows dependence but the table is independent, we can SOMETIMES assume that the graph is not an I-map of the table.
However, if the graph represents a non-set, which belongs in every mathematical set, then it still can be classed as an I-map.