Equations

Question 1

conditional probability, p(a|b) =

Answer 1

p(a,b) / p(b)

Question 2

bayes, p(a|b) =

Answer 2

p(b|a)p(a) / p(b)

Question 3

independent events, p(a,b) =

Answer 3

p(a)p(b)

Question 4

total probability/marginalisation, p(X=x) =

Answer 4

sumy: p(x|y)p(y)

Question 5

conditional independence assumption, p(x|y) =

Answer 5

multiply: p(x|y)

Question 6

discriminant function, f(x) =

Answer 6

sum: wx - t

Question 7

perceptron update rule, sigmoid error wj =

Answer 7

wj - (lrate)(f(x)-y)(x)

Question 8

sigmoid/logistic regression, f(x) =

Answer 8

1 / (1+e^-z)

Question 9

log loss/cross entropy loss, L(f(x),y) =

Answer 9

-{ylogf(x) + (1-y)log(1-f(x))}

Question 10

summed log loss/ cross entropy error/ negative log likelihood, E =

Answer 10

• sum i: {ylogf(x) + (1-y)log(1-f(x))}

Question 11

partial derivative of cross entropy error, dE/dw =

Answer 11

sum: (f(x) - y))(x)

Question 12

partial derivative of sigmoid, dy/dz =

Answer 12

y(1-y)

Question 13

partial derivative of cross entropy error, dE/df(x)

Answer 13

-[y(1/f(x)) - (1-y)(1/(1-f(x)))]

Question 14

specificity =

Answer 14

TN / (FP+TN)

Question 15

precision = positive predictive value =

Answer 15

TP / (TP + FP)

Question 16

recall = sensitivity = tp rate =

Answer 16

TP / P

Question 17

fp rate =

Answer 17

FP / N

Question 18

f1 measure =

Answer 18

2 / (1/precision) + (1/recall)

Question 19

pearsons correlation coefficient =

Answer 19

sum:(x-xhat)(y-yhat) / sqrt(sum:(x-xhat)^2)sum:(y-yhat)^2))

Question 20

information gain/ mutual information =

Answer 20

I(X;Y) = H(Y) - H(Y|X)

Question 21

euclidean distance =

Answer 21

sqrt(sum:(x1-x2)^2)

Question 22

hamming distance =

Answer 22

sum: delta(xi not equal xj)

Question 23

neuron, y(x,w) =

Answer 23

f(wx + b)

Question 24

softmax =

Answer 24

e^z / sumk: e^z

Question 25

gradient descent, wnew =

Answer 25

wold - (lrate)(dL/dw)

Question 26

mean squared error loss, MSE =

Answer 26

1/n sum: (y-t)^2

Question 27

neuron gradient, with sigmoid loss, and squared loss, dL/dw =

Answer 27

dL/dy dy/dz dz/dw = (y-yhat)(yhat)(1-yhat)(x)

Question 28

entropy, H(X) =

Answer 28

sum: p(x)logp(x)

Question 29

sum: p(x)logp(x)

Answer 29

entropy

Question 30

L = 0.5(y-t)^2

Answer 30

squared error loss

Question 31

e^z / sumk: e^z

Answer 31

softmax

Question 32

information gain =

Answer 32

I(X;Y) = H(Y) - H(Y|X)

Question 33

mutual information =

Answer 33

I(X;Y) = H(Y) - H(Y|X)

Question 34

recall =

Answer 34

TP / P

Question 35

sensitivity =

Answer 35

TP / P

Question 36

tp rate =

Answer 36

TP / P

Question 37

precision =

Answer 37

TP / (TP + FP)

Question 38

positive predictive value =

Answer 38

TP / (TP + FP)

Question 39

- sum i: {ylogf(x) + (1-y)log(1-f(x))}

Answer 39

summed log loss/ cross entropy error/ negative log likelihood

Question 40

summed log loss =

Answer 40

- sum i: {ylogf(x) + (1-y)log(1-f(x))}

Question 41

cross entropy error =

Answer 41

- sum i: {ylogf(x) + (1-y)log(1-f(x))}

Question 42

negative log likelihood =

Answer 42

- sum i: {ylogf(x) + (1-y)log(1-f(x))}

Question 43

log loss, L(f(x),y) =

Answer 43

-{ylogf(x) + (1-y)log(1-f(x))}

Question 44

cross entropy loss, L(f(x),y) =

Answer 44

-{ylogf(x) + (1-y)log(1-f(x))}

Question 45

sigmoid, f(x) =

Answer 45

1 / (1+e^-z)

Question 46

logistic regression, f(x) =

Answer 46

1 / (1+e^-z)

Question 47

bias update for logistic regression, t =

Answer 47

t + lrate(f(x) - y)

Question 48

bias update for perceptron, t =

Answer 48

t = t +lrate(yhat - y)

Question 49

what is P(A or B) if a) they are disjoint b) they are joint

Answer 49

a) P(A) + P(B) | b) P(A) + P(B) - P(A and B)

Question 50

give the bernoulli distribution

Answer 50

``` P(X = 0) = 1 - p p(X = 1) = p ```

Question 51

give the binomial distribution

Answer 51

P(X = k) = (nCk)(p^k)(1-p)^k

Question 52

give the geometric distribution

Answer 52

P(X=x) = (1-p)^x-1 (p)

Question 53

give the poisson distribution

Answer 53

P(X=x) = { lambda^x e(-lambda) } / x!

Question 54

if a discrete r.v. X has a pmf f(X) what is the expected value E[g(x)]

Answer 54

sum i: g(Xi)f(Xi)

Question 55

if a discrete r.v. X has a pmf f(X) what is the variance V[g(x)]

Answer 55

E[(g(X) - E(g(X)))^2] = E[g(X)^2] - E[g(X)]^2

Question 56

properties of Expectations | E[aX + b] =

Answer 56

aE[X] + b

Question 57

properties of variance: | V[aX+b] =

Answer 57

a^2V[X]

Question 58

give the equation for hinge loss

Answer 58

sum: -y(wx + b) = sum: -y(yhat) sum all the negative values for ONLY the misclassified samples

Question 59

when we perform minibatch sgd, what do we times sum:dL/dW by to scale it

Answer 59

n / |S| | n samples / batch size

Question 60

what is the perceptron weight update, with hinge loss?

Answer 60

wj = wj - (lrate)( - yhat x y)(xj) or if just for the misclassified wj = wj - (lrate)( -y)(xj) = wj + (lrate)(y)(xj)

Question 61

what is the loss function (negative log-likelihood) for SGD for logistic regression

Answer 61

- 1/n sumi->n:[yi log f(xi) + (1-yi) log (1-f(xi))] same but with 1/n to rescale based on sample size

Question 62

the decision boundary for logistic regression is given by

Answer 62

d = 1 / (1+e^-z) wx + b = log(d / 1-d)

Question 63

give the equation for zero mean, unit variance normalisation

Answer 63

(x - x_mean) / sigma

Question 64

give the equation for restrict range normalisation

Answer 64

- (x - x_min) / (x_max - x_min)

Question 65

give the equation for fisher score, F=

Answer 65

(m1 - m2)^2 ---------------- v1 + v2

Question 66

give a kernel for horizontal lines

Answer 66

1 1 1 0 0 0 -1 -1 -1

Question 67

give a kernel for vertical lines

Answer 67

1 0 -1 1 0 -1 1 0 -1

Question 68

give the distribution update scheme for adaboost, i.e. what do we multiply Dj(i) by

Answer 68

1 / 2ej if the classification was incorrect | 1 / 2(1-ej) if the classification was correct

Question 69

if we know that A is conditionally independent of B given C, then P(A|B,C) = ?

Answer 69

P(A|C)

Question 70

if A is conditionally independent of B given C, then P(A|B,C) = P(A|C), prove it

Answer 70

P(A,B|C) = P(A|C)P(B|C), conditional independence P(A,B,C) / P(C) = P(A,C)/P(C) P(B|C)/P(C), conditional probability P(A,B,C) = P(A,C)P(B,C) / P(C), times by P(C) P(A,B,C)/P(B,C) = P(A,C)/P(C), divide by P(B,C) PA|B,C) = P(A|C)

Question 71

if A and B are conditionally independent given C then we know?

Answer 71

P(A,B|C) = P(A|C)P(B|C)

Question 72

d e^x / dx = ?

Answer 72

x' e^x

Question 73

d ln x/ dx = ?

Answer 73

1 / x

Question 74

product rule, d(uv) / dx

Answer 74

u dv/dx + v du/dx

Question 75

d log f(x) / dx =

Answer 75

f'(x) / f(x)