KNN

Question 1

What type of model is k-NN?

Answer 1

Non-parametric.

Question 2

Does k-NN train model parameters like θ?

Answer 2

No, it stores the training data instead.

Question 3

How does k-NN make predictions for classification?

Answer 3

By majority vote of the k closest training points.

Question 4

How does k-NN make predictions for regression?

Answer 4

By averaging the outputs (y) of the k nearest points.

Question 5

What does a small k value do in k-NN?

Answer 5

Makes the model highly flexible and sensitive to noise.

Question 6

What happens when k is too small in k-NN?

Answer 6

The model overfits (low bias, high variance).

Question 7

What happens when k is too large in k-NN?

Answer 7

The model underfits (high bias, low variance).

Question 8

What is the most common strategy for handling ties in k-NN classification?

Answer 8

Use odd values of k to reduce tie probability.

Question 9

What is the Euclidean distance formula?

Answer 9

√Σ(xᵢ - yᵢ)²

Question 10

What is the Manhattan distance formula?

Answer 10

Σ|xᵢ - yᵢ|

Question 11

When is cosine similarity useful in k-NN?

Answer 11

When direction matters more than magnitude (e.g. text data).

Question 12

What is one-hot encoding used for in k-NN?

Answer 12

To handle categorical variables.

Question 13

What distance metric can be used with one-hot encoded features?

Answer 13

Hamming distance.

Question 14

What is Jaccard similarity used for?

Answer 14

Comparing overlap between sets.

Question 15

What is the main drawback of k-NN?

Answer 15

Slow prediction and high memory usage.

Question 16

What do k-NN prediction curves show in classification tasks?

Answer 16

How predicted probabilities change across the input space.

Question 17

What does a sharp transition in a k-NN prediction curve indicate?

Answer 17

The model is highly sensitive to local data; likely overfitting.

Question 18

What does a smooth transition in a k-NN prediction curve indicate?

Answer 18

The model is generalizing better; less variance.

Question 19

What does a very flat k-NN prediction curve suggest?

Answer 19

The model is underfitting; it’s not responsive to class boundaries.

Question 20

How does increasing k affect the shape of the k-NN decision boundary?

Answer 20

It makes the boundary smoother and less sensitive to individual points.

Question 21

How does decreasing k affect the shape of the k-NN decision boundary?

Answer 21

It makes the boundary more complex and sensitive to noise.

Question 22

What is the tradeoff shown in k-NN decision boundary plots?

Answer 22

The bias-variance tradeoff.

Question 23

What kind of bias and variance do small k values typically produce?

Answer 23

Low bias, high variance.

Question 24

What kind of bias and variance do large k values typically produce?

Answer 24

High bias, low variance.

Question 25

Why are odd values of k often preferred in classification?

Answer 25

To avoid tie votes between classes.

Question 26

How is the best k usually chosen?

Answer 26

Using cross-validation.

Question 27

What is a key advantage of k-NN?

Answer 27

It is simple and easy to implement.

Question 28

Does k-NN require training?

Answer 28

No, it is a lazy learner that defers computation to prediction time.

Question 29

Can k-NN adapt to complex decision boundaries?

Answer 29

Yes, especially with small k values.

Question 30

Does k-NN work with both classification and regression?

Answer 30

Yes, it supports both.

Question 31

Why is k-NN considered non-parametric?

Answer 31

Because it makes no assumptions about the data distribution.

Question 32

What is a major disadvantage of k-NN?

Answer 32

Slow predictions, especially on large datasets.

Question 33

Why is k-NN memory-intensive?

Answer 33

It stores the entire training dataset.

Question 34

How does k-NN handle irrelevant features?

Answer 34

Poorly — irrelevant or unscaled features can mislead distance calculations.

Question 35

Is k-NN sensitive to feature scaling?

Answer 35

Yes, features with larger ranges can dominate distance calculations.

Question 36

What happens to k-NN performance in high-dimensional spaces?

Answer 36

It degrades due to the curse of dimensionality.