Week 6 - Machine Learning I

Question 1

Define Artificial Intelligence (AI)

Answer 1

AI is the science and engineering of making intelligent machines

Question 2

Define Machine Learning (ML)

Answer 2

ML is a computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.

Question 3

Define Deep Learning

Answer 3

Deep Learning uses computational structures known as ‘neural networks’ to automatically recognise patterns in data and provide a suitable output, such as a prediction or evidence for a decision

Question 4

Define Natural Language Processing (NLP)

Answer 4

NLP could be as simple as counting word frequencies to compare different writing styles, to ‘understading’ complete human utterances

Question 5

What are the types of machine learnings?

Answer 5

1. Supervised learning
2. Unsupervised learning
3. Reinforcement learning

Question 6

What is supervised machine learning?

Answer 6

Supervised machine learning takes a set of input features (predictors) and output variables (i.e., labelled) and learns a mapping function between input and output

Question 7

What is unsupervised machine learning?

Answer 7

Unsupervised machine learning learns patterns from unlabeled data (without the explicit output variable)

Question 8

What is reinforcement machine learning?

Answer 8

The agent (learning system) learns to perform a task by interacting with an unknown environment and learns a policy that maximises the reward. Dialogue generation and machine translation are application areas in NLP for reinforcement learning.

Question 9

What is the typical process of Machine Learning?

Answer 9

1. Obtains the data
2. Goes through preprocessing
3. Input features
4. Output
5. Machine learning

Question 10

What are examples of preprocessing?

Answer 10

Preprocessing includes:
1. converted text to lower case
2. special characters and stopwords are removed
3.removal of URLSs, numbers, punctuation, stopwords, whitespace and stemming

Question 11

What are examples of input features?

Answer 11

Input features would include:
1. Bag of Words
2. TF - IDF
3. TF - IDF with ngrams

Question 12

What are examples of output features?

Answer 12

Output features would include:
1. Recruitment and not-recruitment
2. Predatory and non-predatory

Question 13

What are examples of Machine Learning processes?

Answer 13

1. Naive Bayes
2. Support vector machine
3. Logistic regression
4. Neural networks

Question 14

How many sub-classes of supervised learning are there?

Answer 14

There are two.

Classification - training dataset with input features and discrete output (target or class variables). The goal of classification is to learn the mapping function to map the input to the discrete output

Regression - predict the real or continuous value of output (target variable) given a set of input features called predictors

Question 15

What classifies as binary classification and multiclass classification?

Answer 15

Binary classification - deals with two possible classes (e.g., spam or not spam, fraudulent or legitimate, hate or not-hate, hate-speech or normal-speech)

Multiclass classification - deals with three or more classes (e.g., fake-news, partially fake, true, unknown // identity theft, cyberstalking, fraudulent sales, legitimate)

Question 16

What’s the key challenge of supervised learning?

Answer 16

1. learn from the data (training data)
2. How does it work on ‘unseen’ data

Question 17

What is a optimal model in ML terms and what are its key challenges?

Answer 17

1. Underfitting - too simple model, and it fails to capture the relationship between input and output variables. High training error on training and high test error on unseen (new) data
2. Overfitting - low training error and high test error
3. Optimal fitting - low training error and low test error

Question 18

What is the process of applied machine learning in practice 1?

Answer 18

1. Problem statement - identify the appropriate task for ML
2. Data acquisition
3. Clean and pre-process data

Question 19

What is the process of applied machine learning in practice 2?

Answer 19

1. Feature engineering and selection of features
2. Select appropriate machine learning algorithms
3. Train the model
4. Model validation and evaluation: tune the hyperparameters and evaluate the performance
5. Interpret the results so you can deploy the model

Question 20

How can you choose the best algorithm for the task?

Answer 20

You can use the algorithm cheat-sheet in which it looks at which area of data it should be in: classification/ clustering/ regression/ dimensionality reduction

Question 21

What is Naive Bayes?

Answer 21

Naive Bayes is a simple and fast supervised machine learning algorithm used for classification (binary and multiclass).

This is known as a probabilistic classifier and applies the bayes theorem.

Question 22

What is Support Vector Machine (SVM)?

Answer 22

SVM is used for classification, regression and outlier detection (one class SVMs) and it performs well for complex (high-dimensional feature) and on small-medium size datasets

Question 23

What is the goal of SVM?

Answer 23

SVM’s goal is to have the maximum margin hyperplane that provides the largest distance between the two classes and new observations are predicted based on the side of the hyperplane they fall to

Question 24

How to test if the model in R performs ell on unseen (new) data?

Answer 24

You should split your dataset into training and test sets. Then you test the performance of the model on the new data.

Question 25

How do you split the dataset into train and test sets?

Answer 25

If we have a large dataset (e.g., 500,000) we can choose a smaller split, but with smaller datasets, we may need to choose a larger split for the test set

Question 26

Why is it important to test the model?

Answer 26

Testing the model to determine how well it generalises to new cases using test data

Question 27

What are the issues with single-train test splits?

Answer 27

1. Model performance could be really dependent on the training data 2. Would like to be more careful 3. Cross-validation techniques can be used to validate the generalisation ability of the machine learning models 4. Cross-validation can be used on binary and multi-classification tasks

Question 28

What is k-fold cross validation?

Answer 28

K-fold cross validation is one of the most widely used cross validation techniques for classification models. It splits the data randomly into k number of folds and at each iteration (from 1-k) use k-1 folds for training and remaining 1-fold is used for testing Normally this is done on training data, and make a final evaluation on an unseen test set

Question 29

How can you compare the performance of multiple classifiers?

Answer 29

You can use the following performance metrics: 1. Precision/recall trade-off 2. F1 Score (F-score/F-measure) 3. Recall metric 4. Precision metric 5. Accuracy metric 6. Confusion matrix 7. ROC (receiver operating characteristics) 8. AUC (area under the curve)

Question 30

What is performance metrics?

Answer 30

The metrics used for measuring and evaluating the performance of machine learning models

Question 31

What is the confusion matrix?

Answer 31

The confusion matrix is a table that helps to better visualize the performance of classifiers. More concise metrics can be computer from the confusion matrix (accuracy, precision, recall and f1-score). Look at nb for matrix

Question 32

What is the accuracy metric?

Answer 32

Using the confusion matrix, you can calculate the accuracy of a classifier as: accuracy = (TP + TN)/ (TP + FP + FN + TN) This is a widely used metric but can be misleading if used on the imbalanced dataset (a common situation for real-world problems) Note: avoid using accuracy metrics alone for classification problems

Question 33

What is the precision metric?

Answer 33

The ratio of correctly classified observations to the total observations predicted to be positive. Usually they emphasise the accuracy of positive predictions. precision = TP / (TP + FP)

Question 34

What is the recall metric?

Answer 34

The recall metric is the ratio of correctly classified observations to the total observations that are in fact positive. This is also known as sensitivity. recall = TP/TP + FN

Question 35

What is F1 Score (F-score/F-measure)?

Answer 35

F1 score incorporates both recall and precision. The harmonic mean of the precision and recall will (1) give more weight to lower values than the arithmetic mean (2) F1 is high if both the recall and precision are high F1 score = 2*(precision*recall)/precision + recall

Question 36

What is the relationship for a Recall Trade-off?

Answer 36

By increasing the precision reduces recall and vice versa, known as the precision/recall trade-off. Hence, we can denote the relationship as: Classifier A: low recall, higher precision Classifier B: high recall, lower precision

Question 37

What is Receiver Operating Characteristics (ROC)?

Answer 37

This is used to show the performance of binary classification. This displays the tradeoff between True Positive Rate (TRP) and FPR (False Positive Rate) for various thresholds

Question 38

Define and formulate: TPR and FPR

Answer 38

TPR (sensitivity, recall): ratio of positive classes that were classified correctly sensitivity = TP / TP + FN FPR (1-specificity): specificity is the ratio of the negative class that were classified correctly specificity = TN / TN + FP

Question 39

What is class probabilities?

Answer 39

Class probabilities is a threshold that can be used to assign probability values to classes (obtain the score for the ROC curve). Each instance in the classifier has a class probability (between 0 and 1). A representation (visual) is good to show all the possible values across.

Question 40

Which classifier is better? Do this in terms of AUC.

Answer 40

Measure the area under the curve (AUC) which is a value between 0 and 1. A perfect classifier will have an AUC = 1 Purely random classifier will have an AUC = 0.5