K-NN

Question 1

How do we classify a point using K-NN?

Answer 1

Look at the k nearest points. Assign the most common class.

Question 2

How do we use K-NN for regression?

Answer 2

Look at the K nearest neighbours, and then take the average of those neighbours.

Question 3

What is the L2 Norm?

Answer 3

The Euclidean Distance

Question 4

What can we use instead of Euclidean Distance for K-NN?

Answer 4

Similarity Measures

• Inner Product
• Cosine Similarity

Question 5

What is the Minkowski distance?

Answer 5

A generalisation of the euclidean distance,

where instead of squaring and square rooting

we use t

so

(|a-b|^t) ^ (1/t)

this extends to higher dimensions

Question 6

What is the Hyper-parameter in K-NN?

Answer 6

The neighbour number K

Question 7

For a classification using N classes, what choices of K should we avoid when using K-NN?

Answer 7

We should never set K to equal aN

Setting K to be a multiple of N may result in ties, making it impossible to come to a decision.

Question 8

What may be the consequence of choosing a small K for K-NN?

Answer 8

We may model noise

Question 9

What may be the consequence of choosing a large K for K-NN?

Answer 9

We may include too many samples from other classes,

which can negatively affect our prediction.

Gives us an inaccurate prediction.

Question 10

How many parameters are used in K-NN?

Answer 10

0 parameters used.

This is a non-parametric model.