Chapter 4

Question 1

In classification, a Type I error, also known as a ______, occurs when a patient who does not have a disease receives a positive diagnostic.

Answer 1

false positive

Question 2

A Type II error, or ______, happens when a patient has a disease but it was not detected.

Answer 2

false negative

Question 3

The four fundamental outcomes in a binary classification task are True Positive (TP), True Negative (TN), ______, and ______.

Answer 3

False Positive (FP),False Negative (FN)

Question 4

In a confusion matrix, the sum of True Positives and False Positives (TP + FP) represents the ______.

Answer 4

Total Predicted Positives (RP)

Question 5

The formula for Accuracy in classification is (TP + TN) / ______, where P is total actual positives and N is total actual negatives.

Answer 5

P + N

Question 6

Precision, defined as TP / (TP + FP), measures the proportion of ______ among the instances predicted as positive.

Answer 6

correctly predicted positive instances

Question 7

Recall, also known as sensitivity or True Positive Rate, is calculated as TP / (TP + FN) and measures the proportion of ______ that were correctly identified.

Answer 7

actual positive instances

Question 8

The F1-score is the harmonic mean of ______ and ______, providing a balance between them.

Answer 8

Precision,Recall

Question 9

Sensitivity, calculated as TP / (TP + FN), can be thought of as the likelihood of spotting a ______ when presented with one.

Answer 9

positive case

Question 10

Specificity, calculated as TN / (TN + FP), is the likelihood of spotting a ______ when presented with one.

Answer 10

negative case

Question 11

The Receiver Operating Characteristic (ROC) curve plots the True Positive Rate (TPR) against the ______ at various threshold settings.

Answer 11

False Positive Rate (FPR)

Question 12

The True Positive Rate (TPR) used in ROC analysis is also known as ______ or ______.

Answer 12

recall,sensitivity

Question 13

The False Positive Rate (FPR) used in ROC analysis, calculated as FP(t)/N, is also referred to as ______ or ______.

Answer 13

fallout,false alarm rate

Question 14

To compare different classification algorithms using ROC curves, one typically compares their ______, often abbreviated as AUC.

Answer 14

Area Under the Curve

Question 15

Categorical data values that are simply names or labels with no ordering defined, such as gender or color, are known as ______ values.

Answer 15

Nominal

Question 16

Categorical data values where order does matter, such as t-shirt size or rank, are called ______ values.

Answer 16

Ordinal

Question 17

The process of replacing each category in categorical data with a unique number is known as ______.

Answer 17

String Indexing

Question 18

______ is a technique used to break the inherent ordering within a categorical column by creating new binary columns for each unique category.

Answer 18

One Hot Encoding (OHE)

Question 19

The process that aims to optimize a model’s configuration to achieve the best possible performance for a specific problem is called ______.

Answer 19

Hyperparameter Tuning

Question 20

Common techniques for hyperparameter optimization include Manual Search, Grid Search, Random Search, and ______.

Answer 20

Bayesian Optimization

Question 21

In regression tasks, the ______ is a common metric calculated as the square root of the Mean Squared Error (MSE).

Answer 21

root mean squared error (RMSE)

Question 22

The Mean Squared Error (MSE) indicates how close a regression line is to a set of data points by taking the distances or ‘errors’ from the points to the regression line and ______ them.

Answer 22

squaring

Question 23

A common practice when handling numerical attributes is to assume ______ for these attributes.

Answer 23

normal distributions

Question 24

Error on the ______ is not a good indicator of a model’s performance on future data.

Answer 24

training data

Question 25

To properly tune parameters and evaluate a model, one should ideally use three distinct datasets: training data, ______, and test data.

Answer 25

validation data, test data

Question 26

The ______ procedure involves splitting the original data into a training set and a test set, for example, reserving 2/3 for training and 1/3 for testing.

Answer 26

Holdout

Question 27

______ is a method where the data is partitioned into k disjoint subsets, and the model is trained on k-1 partitions and tested on the remaining one, repeating this k times.

Answer 27

K-fold Cross-validation

Question 28

A disadvantage of the Holdout method is that its evaluation can have a ______, heavily depending on which data points end up in the training and test sets.

Answer 28

high variance

Question 29

In k-fold cross-validation, every data point gets to be in a test set exactly once and gets to be in a training set ______ times.

Answer 29

k-1

Question 30

A computationally expensive form of cross-validation where the number of folds is equal to the number of training instances is called ______ cross-validation.

Answer 30

Leave-One-Out

Question 31

The performance of a machine learning algorithm on unseen data is referred to as its ______.

Answer 31

generalization error

Question 32

______ is a resampling technique used to estimate the distribution of a dataset by sampling with replacement from the original dataset.

Answer 32

Bootstrap

Question 33

The difference between prediction values made by a model and the actual or expected values is known as ______ or Errors due to bias.

Answer 33

bias errors

Question 34

A model with ______ makes more assumptions and is unable to capture important features of the dataset, performing poorly on new data.

Answer 34

high bias

Question 35

______ errors indicate how much a random variable's prediction differs from its expected value, with high values suggesting the model doesn't generalize well from training data.

Answer 35

Variance

Question 36

A model that shows ______ learns the training dataset very well but performs poorly on unseen datasets, indicating overfitting.

Answer 36

high variance

Question 37

To reduce ______, one might increase the number of input features, decrease regularization, or use more complex models.

Answer 37

high bias

Question 38

To reduce ______, one might reduce the number of input features, use a less complex model, increase training data, or increase regularization.

Answer 38

high variance

Question 39

In Scikit-learn, the ______ argument in functions like RandomForestClassifier allows you to specify the number of CPU cores to use for computationally expensive tasks.

Answer 39

n_jobs

Question 40

Setting `n_jobs=-1` in Scikit-learn tasks will typically instruct the process to use ______ available on the system.

Answer 40

all of the cores

Question 41

CPU technology such as ______ can allow a single physical CPU core to function as two logical cores, effectively doubling the number of cores available for parallel processing.

Answer 41

hyper-threading

Question 42

The classification result FP stands for ______, representing a Type I error.

Answer 42

false positive

Question 43

The classification result FN stands for ______, representing a Type II error.

Answer 43

false negative

Question 44

The sum $TN+TP+FN+FP$ in a classification context represents the ______.

Answer 44

total number of instances

Question 45

In a confusion matrix, $P = TP+FN$ represents the total number of ______ instances.

Answer 45

actual positive

Question 46

In a confusion matrix, $N = TN+FP$ represents the total number of ______ instances.

Answer 46

actual negative

Question 47

The F1-score is specifically the ______ of Precision and Recall.

Answer 47

Harmonic mean

Question 48

An ROC curve for a ______ would go straight up the Y-axis to (0,1) and then straight across to (1,1).

Answer 48

perfect classifier

Question 49

An ROC curve for a ______ would typically be a diagonal line from (0,0) to (1,1).

Answer 49

random classifier

Question 50

When using One-Hot Encoding, if a categorical feature has 'n' unique categories, it will be transformed into ______ new binary columns.

Answer 50

n

Question 51

Hyperparameters are set ______ training a model, unlike parameters which are learned ______ training.

Answer 51

before

Question 52

The Mean Absolute Error (MAE) is calculated by taking the average of the ______ differences between predicted and actual values.

Answer 52

absolute

Question 53

Error rate obtained by evaluating a model on the ______ can be misleadingly optimistic.

Answer 53

training data

Question 54

The main purpose of a ______ is to tune hyperparameters without touching the final test set.

Answer 54

validation data

Question 55

Stratified k-fold cross-validation ensures that each fold is ______ of the overall dataset in terms of class distribution.

Answer 55

representative

Question 56

Bootstrap resampling is done ______ from the original dataset.

Answer 56

with replacement

Question 57

Underfitting is often characterized by ______ and ______, while overfitting is characterized by ______ and ______.

Answer 57

high bias, low variance, low bias, high variance

Question 58

The three main computationally expensive centers in machine learning are training models, evaluating models, and ______.

Answer 58

hyperparameter tuning machine learning models

Question 59

If a CPU has 4 physical cores and supports hyper-threading, it can present as ______ logical cores to the operating system.

Answer 59

eight

Question 60

A message that is not spam but is assigned to the spam folder is an example of a ______ error in spam detection.

Answer 60

Type I

Question 61

A message that is spam but appears in the regular folder is an example of a ______ error in spam detection.

Answer 61

Type II

Question 62

The term 'fallout' in the context of ROC analysis refers to the ______.

Answer 62

False Positive Rate (FPR)

Question 63

If a model has low bias, it makes ______ about the form of the target function.

Answer 63

fewer assumptions

Question 64

If a model has high variance, it shows a ______ in the prediction of the target function with changes in the training dataset.

Answer 64

large variation

Question 65

Using a ______ complex model can help reduce high bias but may increase variance.

Answer 65

more

Question 66

Increasing the ______ term is a common strategy to reduce high variance (overfitting).

Answer 66

regularization

Question 67

When evaluating models, if labeled data is scarce, techniques like ______ are preferred over a simple holdout split.

Answer 67

cross-validation

Question 68

The primary goal of hyperparameter tuning is to find a set of hyperparameters that results in the best ______ on unseen data.

Answer 68

generalization performance

Question 69

In the context of performance metrics, $RP = TP+FP$ represents the ______ by the classifier.

Answer 69

total instances predicted as positive

Question 70

In the context of performance metrics, $RN = TN+FN$ represents the ______ by the classifier.

Answer 70

total instances predicted as negative

Question 71

The formula for Error rate is ______ / (P+N).

Answer 71

FP+FN

Question 72

A key assumption for both training and test data is that they are ______ of the underlying problem.

Answer 72

representative samples

Question 73

If a learning scheme operates in two stages, the first stage typically involves building the ______ of the model.

Answer 73

basic structure

Question 74

Generally, the larger the training data the better the classifier, although this effect often shows ______.

Answer 74

diminishing returns

Question 75

The dilemma in splitting data is that ideally, both the training set and the test set should be ______.

Answer 75

large

Question 76

Cross-validation avoids ______ test sets, which can be an issue with repeated holdout methods.

Answer 76

overlapping

Question 77

Leave-One-Out cross-validation makes the best use of the data and involves no ______.

Answer 77

random subsampling

Question 78

A ______ is a subset of available data that is not used for training the model but is used to estimate the model's performance on unseen data during development.

Answer 78

validation set

Question 79

High bias can occur if the model is too ______ for the complexity of the data.

Answer 79

simple

Question 80

A model that performs well on the training data but poorly on the test data is said to be ______.

Answer 80

overfitting

Question 81

The `n_jobs` parameter in scikit-learn is used to enable ______ for certain operations.

Answer 81

parallel computation

Question 82

The process of breaking a categorical column into n different columns, where n is the number of unique categories, is the first step of ______.

Answer 82

One Hot Encoding

Question 83

Sensitivity is the proportion of ______ we find.

Answer 83

edges (positive cases)

Question 84

Specificity is the proportion of ______ that we find.

Answer 84

non-edges (negative cases)

Question 85

A perfect classifier on an ROC curve would have an AUC value of ______.

Answer 85

1

Question 86

A random classifier on an ROC curve would have an AUC value of approximately ______.

Answer 86

0.5

Question 87

The choice of performance measure in evaluation can include the number of correct classifications, accuracy of probability estimates, or ______.

Answer 87

error in numeric predictions

Question 88

When tuning hyperparameters, the ______ is used to optimize these settings, while the ______ provides the final, unbiased performance estimate.

Answer 88

validation data, test data

Question 89

The term ______ refers to the model's inability to capture the underlying trend of the data, often due to an overly simplistic model.

Answer 89

underfitting

Question 90

The term ______ refers to a model that learns the training data too well, including its noise and outliers, leading to poor performance on new data.

Answer 90

overfitting

Question 91

Reducing the number of ______ or parameters can be a way to combat high variance.

Answer 91

input features

Question 92

Increasing the amount of ______ can sometimes help reduce high variance.

Answer 92

training data

Question 93

The scikit-learn library provides the `n_jobs` argument to control the use of ______ for tasks like model training and hyperparameter tuning.

Answer 93

multiple cores

Question 94

Hyper-threading is a technology that allows each physical CPU core to handle ______ simultaneously.

Answer 94

multiple threads (typically two)

Question 95

The total number of instances in a dataset can be represented by the sum ______.

Answer 95

TP + TN + FP + FN

Question 96

The marginal P in a confusion matrix, calculated as TP + FN, represents the total number of ______ instances in the dataset.

Answer 96

actual positive

Question 97

The marginal N in a confusion matrix, calculated as TN + FP, represents the total number of ______ instances in the dataset.

Answer 97

actual negative

Question 98

The marginal RP in a confusion matrix, calculated as TP + FP, represents the total number of instances ______ by the model.

Answer 98

predicted as positive

Question 99

The marginal RN in a confusion matrix, calculated as TN + FN, represents the total number of instances ______ by the model.

Answer 99

predicted as negative

Question 100

If a model consistently makes large errors for a specific subgroup of data, it might be suffering from high ______ for that subgroup.

Answer 100

bias

Question 101

If a model's performance changes drastically with small changes in the training data, it likely has high ______.

Answer 101

variance

Question 102

______ search for hyperparameter tuning explores pre-defined points in the hyperparameter space.

Answer 102

Grid

Question 103

______ search for hyperparameter tuning samples hyperparameter combinations randomly from specified distributions.

Answer 103

Random

Question 104

One advantage of random search over grid search is that it can be more efficient in finding good hyperparameters when some hyperparameters are ______ than others.

Answer 104

more important

Question 105

The evaluation of a model on the ______ is used to estimate its performance on unseen data.

Answer 105

test set

Question 106

When using k-fold cross-validation, the model is trained ______ times.

Answer 106

k

Question 107

Bootstrap involves resampling ______ from the original dataset to create multiple bootstrap samples.

Answer 107

with replacement

Question 108

A model that is underfitting has ______ bias and ______ variance.

Answer 108

high, low

Question 109

A model that is overfitting has ______ bias and ______ variance.

Answer 109

low, high

Question 110

The `n_jobs` parameter in scikit-learn, when set to -1, typically means to use all ______ CPU cores.

Answer 110

available