Topic 17

Question 1

Bayes Theorem

Answer 1

P(Y|X) = P(X|Y)P(Y) /P(X) Where P(Y|X) is the posterior probability (description given all the available data), P(X|Y) is the likelihood (predicts the data), P(Y) is the prior probability (based on prior information) and P(X) is the normalising constant

Question 2

Product Rule for Independent Events

Answer 2

P(X|Y) = P(x1, x2, … , xn|Y) = P(x1|Y)P(x2|Y)…P(xn|Y)

Question 3

Conditional Probability

Answer 3

The probability that A and B are true is the same as the probability that A is true and that B is true given that A is true: P(A and B) = P(A) x P(B|A)

Question 4

Naive Bayes General Formula

Answer 4

P(Y|x1, … , xn) ∝ 𝑃(𝑌)ෑ
Σ 𝑖=1,𝑛 𝑃(𝑥𝑖 |𝑌)

Question 5

Y With Maximum Probability

Answer 5

𝑌𝑝𝑟𝑒𝑑 = 𝑎𝑟𝑔𝑚𝑎𝑥𝑦𝑃(𝑌) Σ n, i=1 𝑃(𝑥𝑖 |𝑌)

Question 6

Laplace/ Plus One Smoothing

Answer 6

Adding 1 to all occurences / data points to avoid having a probability of 0 that will ruin Naive Bayes calculations

Question 7

Multinomial Naive Bayes

Answer 7

Document classification (eg. assigning an article to sports, politics etc or spam vs not spam)

Question 8

Bernoulli Naive Bayes

Answer 8

Similar to Multinomial. but the features / predictors are boolean variables (eg. tennis weather example)

Question 9

Gaussian Naive Bayes

Answer 9

When features are not discrete and take up a continuous value (assumed that the values are sampled from a Gaussian distribution), estimate a normal distribution with some mean and SD for each feature

Question 10

Naive Bayes Advantages

Answer 10

*
It is easy and fast to predict a class of test dataset
*
Naïve Bayes classifier performs better compare to other models assuming independence
*
It performs well in case of categorical input variables compared to numerical variables

Question 11

Naive Bayes Disadvantage

Answer 11

Independent Predictor Assumption (variables might not be independent)