Week 1: Data and KNN

Question 1

What is machine learning approach?

Answer 1

Programming an algorithm to automatically learn from data, or from experience, uncover patterns in data, building autonomous agents

Question 2

What should be emphasized in machine learning?

Answer 2

• Predictive performance
• Scalability
• Autonomy

Question 3

Why might you want to use a learning algorithm?

Answer 3

• Hard to code solution by hand (vision, speech)
• System needs to adapt to a changing environment (spam detection)
• Want the system to perform better than human programmers
• Privacy/ fairness (ranking search results)

Question 4

How does machine learning perform compared to humans?

Answer 4

It may perform better or worse than humans

Question 5

Define artificial intelligence

Answer 5

• A subfield of CS that refers to computer programs that can solve problems humans are good at
• E.g vision, natural language

Question 6

Define machine learning

Answer 6

A subfield of AI focused on learning (tuning parameters) from data

Question 7

Define neural networks

Answer 7

Parametric model used in ML loosely based on biological neurons

Question 8

What is deep learning?

Answer 8

Neural networks with multiple layers

Question 9

What is data science?

Answer 9

An emerging field which applies ml techniques to domain-specific problems

Question 10

What are some machine learning domains?

Answer 10

• Computer vision
• Speech recognition
• Natural Language Processing
• Recommender system
• Games

Question 11

Types of machine learning

Answer 11

• Supervised learning
• Semi-supervised learning
• Reinforcement learning
• Unsupervised learning

Question 12

What is supervised learning

Answer 12

• They have labeled examples of the correct behavior
• Predict unknown values of the data using other known data
• Classification (is this A or B?)
• Anomaly detection (is this weird?)
• Regression (how much/ how many)

Question 13

What is semi-supervised learning

Answer 13

Utilizes both labeled and unlabeled data

Question 14

What is reinforcement learning

Answer 14

Learning system which interacts with the world and learns to maximize a scalar reward signal

Question 15

What is unsupervised learning

Answer 15

• No labeled examples, instead looking for interesting patterns in the data
• Find human interpretable and previously unknown patterns that describe the unlabeled data
• Clustering (how is data organized)
• Association rule mining (are these related?)

Question 16

Why is machine learning so powerful nowadays?

Answer 16

• Abundance of data
• Computing power

Question 17

What is the machine learning problem?

Answer 17

• Should I use ml on this problem?
• Gather and organize data (pre-processing, cleaning, visualizing)
• Establish a baseline
• Choosing a model
• Optimization
• Hyperparameter search
• Analyze performance and mistakes
  -Iterate back to step 4 or 2

Question 18

What is data?

Answer 18

Collection of objects and their attributes

Question 19

What does a ml training set consist of?

Answer 19

• Inputs (vectors)
• Labels

Question 20

Why do we use input vectors in machine learning?

Answer 20

• Algorithms need to handle lots of data
• A common strategy is mapping data to another space that is easy to manipulate (Representation)
• Vectors are a good representation since we can do linear algebra

Question 21

What is regression and classification in a training set?

Answer 21

Regression- t is a real number
- Classification- t is an element of a discrete set

Question 22

What are the classification metrics for evaluation?

Answer 22

Accuracy= # correct predictions/ # test instances
Error= 1 - accuracy= # incorrect predictions/ # test instances

Question 23

What is similarity?

Answer 23

• The simplest method of learning we know
• Classifying according to similar objects you’ve seen
• aka manohorse

Question 24

What happens when more data points come in to nearest neighbor?

Answer 24

More complicated boundaries are possible

Question 25

What is nearest neighbors relationship with noise?

Answer 25

It is sensitive to noise or mislabeled data (class noise)

Question 26

What is the solution to noisy data?

Answer 26

Have k-nearest neighbors vote and pick the majority

Question 27

What are the steps for k-nearest neighbors?

Answer 27

- Calculate the distance between the new data point and all the datapoints in the set - Identify the k points with the shortest distance to the new point, these are the k nearest neighbors - Among the nearest neighbors, count how many points there are for each class type and pick the majority

Question 28

What is k?

Answer 28

K determines the tradeoff between fitting the data and overfitting the data

Question 29

What happens when there is a small k?

Answer 29

- Good at capturing fine-grained patterns - May overfit, sensitive to local variations in training data

Question 30

What happens when there is a large k?

Answer 30

- Makes stable predictions by averaging over lots of examples - May underfit because model is too generalized and oversimplifies underlying patterns in the data

Question 31

How do you balance k?

Answer 31

- Optimal k depends on the number of datapoints (n) - As a rule of thumb, choose k=3 - k < root n

Question 32

What is validation set used for?

Answer 32

Tuning hyperparameters

Question 33

What is cross validation?

Answer 33

Used to estimate generalization error of a learning algorithm when the given dataset is too small for a simple train/test or train/valid split to yield accurate estimation of generalization error

Question 34

What is k-fold cross validation?

Answer 34

- A partition of dataset is formed by splitting it into k non-overlapping subsets - Estimate the test error by taking the average test score across k trials - On trial i, the i-th subset is the test set, the rest is training set

Question 35

What are the highlights of k-nearest neighbor?

Answer 35

- Simple - No training - Easy to justify classification to customer - Can easily do multiclass

Question 36

What are the limitations of KNN? Large dataset

Answer 36

- Lazy learning technique - in training phase KNN doing nothing, so training is fast - in time of prediction it becomes slow as large dataset comes since model has to calculate Euclidean distance from given point to all points in the dataset

Question 37

What are the limitations of KNN? Curse of Dimensionality

Answer 37

- feature space becomes increasingly sparse as the number of dimensions (features) grows - In high-dimensional spaces, the notion of proximity or similarity becomes less meaningful

Question 38

What are the limitations of KNN? Imbalanced dataset

Answer 38

- the majority class typically has significantly more samples than the minority class. - large number of neighbors from the majority class can overpower the neighbors from the minority class - dominate the decision making process, leading to a bias towards the majority class in the predictions.