Machine Learning

Question 1

What is Semi-supervised Machine Learning?

Answer 1

Semi-supervised learning is the blend of supervised and unsupervised learning. The algorithm is trained on a mix of labeled and unlabeled data. Generally, it is utilized when we have a very small labeled dataset and a large unlabeled dataset.

In simple terms, the unsupervised algorithm is used to create clusters and by using existing labeled data to label the rest of the unlabelled data. A Semi-supervised algorithm assumes continuity assumption, cluster assumption, and manifold assumption.

It is generally used to save the cost of acquiring labeled data. For example, protein sequence classification, automatic speech recognition, and self-driving cars.

Question 2

What is the manifold assumption in semi-supervised learning?

Answer 2

The manifold assumption in semi-supervised learning states that
(a) the input space is composed of multiple lower-dimensional manifolds on which all data points lie and
(b) data points lying on the same manifold have the same label

Question 3

What is the continuity assumption in semi-supervised learning?

Answer 3

The continuity assumption states that objects near each other tend to share the same group or label.

This assumption is also used in supervised learning, and the datasets are separated by the decision boundaries.

In semi-supervised learning, the decision boundaries are added with the smoothness assumption in low-density boundaries.

Question 4

What is the cluster assumption in semi-supervised learning?

Answer 4

The cluster assumption states that data are divided into different discrete clusters, and that points in the same cluster share the output label.

Question 5

How do you choose which algorithm to use for a dataset?

Answer 5

This will depend on the business use case, amounts of labelled data and application requirements

Question 6

What is supervised machine learning?

Answer 6

Supervised ML is when the algorithm is trained using a labeled dataset, which consists of pairs of input and output data.

The main classes are Regression and Classification

Question 7

What are regression based algorithms?

Answer 7

In regression, the target variable is a continuous value. The goal of regression is to predict the value of the target variable based on the input variables.

Linear regression, polynomial regression, and decision trees are some of the examples of regression algorithms.

Question 8

What are classification based algorithms?

Answer 8

In classification, the target variable is a categorical value. The goal of classification is to predict the class or category of the target variable based on the input variables.

Some examples of classification algorithms include logistic regression, decision trees, support vector machines, and neural networks.

Question 9

What is linear regression?

Answer 9

Linear regression is a type of regression algorithm that is used to predict a continuous output value. It is one of the simplest and most widely used algorithms in supervised learning. In linear regression, the algorithm tries to find a linear relationship between the input features and the output value. The output value is predicted based on the weighted sum of the input features.

Question 10

What is logistic regression?

Answer 10

Logistic regression is a type of classification algorithm that is used to predict a binary output variable. It is commonly used in machine learning applications where the output variable is either true or false, such as in fraud detection or spam filtering. In logistic regression, the algorithm tries to find a linear relationship between the input features and the output variable. The output variable is then transformed using a logistic function to produce a probability value between 0 and 1.

Question 11

What are decision trees?

Answer 11

A decision tree is a type of algorithm that is used for both classification and regression tasks.

Consists of three components: decision nodes, leaf nodes, and a root node.

A decision tree algorithm divides a training dataset into branches, which further segregate into other branches. This sequence continues until a leaf node is attained. The leaf node cannot be segregated further.

The nodes in the decision tree represent attributes that are used for predicting the outcome.

It is used to model decisions and their possible consequences.

Each internal node in the tree represents a decision, while each leaf node represents a possible outcome. Decision trees can be used to model complex relationships between input features and output variables.

Question 12

What are random forests?

Answer 12

Random forests are an ensemble learning technique that is used for both classification and regression tasks. They are made up of multiple decision trees that work together to make predictions. Each tree in the forest is trained on a different subset of the input features and data (data bagging). The final prediction is made by aggregating the predictions of all the trees in the forest.

Question 13

Explain the K Nearest Neighbor Algorithm

Answer 13

The K Nearest Neighbor (KNN) is a supervised learning classifier. It uses proximity to classify labels or predict the grouping of individual data points. We can use it for regression and classification. KNN algorithm is non-parametric, meaning it doesn’t make an underlying assumption of data distribution.

In the KNN classifier:

We find K-neighbors nearest to the white point. In the example below, we chose k=5.
To find the five nearest neighbors, we calculate the euclidean distance between the white point and the others. Then, we chose the 5 points closest to the white point.
There are three red and two green points at K=5. Since the red has a majority, we assign a red label to it.

Question 14

Is it true that we need to scale our feature values when they vary greatly?

Answer 14

Yes. Most of the algorithms use Euclidean distance between data points, and if the feature value varies greatly, the results will be quite different. In most cases, outliers cause machine learning models to perform worse on the test dataset.

We also use feature scaling to reduce convergence time. It will take longer for gradient descent to reach local minima when features are not normalized.

Question 15

What are the different types of error present in machine learning?

Answer 15

There are mainly two types:
- Reducible errors - can be reduced to improve model accuracy. Such errors can further be classified into bias and Variance.
- Irreducible errors: These errors will always be present in the model

Question 16

What is the bias/variance trade off?

Answer 16

For an accurate model, algorithms need a low variance and low bias. But this is not possible because bias and variance are related to each other:

Decreasing variance will increase bias
Decreasing bias will increase variance

Ideally, we need a model that accurately captures the regularities in training data and simultaneously generalizes well with the unseen dataset. Unfortunately, doing this is not possible simultaneously.

High bias + low variance = Underfitting
Low bias + high variance = Overfitting

The Bias-Variance trade-off is about finding the sweet spot to make a balance between bias and variance errors.

Question 17

What is bias?

Answer 17

Bias is the difference between the average prediction of our model and the correct value which we are trying to predict. Model with high bias pays very little attention to the training data and oversimplifies the model (underfitted). It always leads to high error on training and test data.

Question 18

What is variance?

Answer 18

Variance is the variability of model prediction for a given data point or a value which tells us spread of our data. Model with high variance pays a lot of attention to training data and does not generalize on the data which it hasn’t seen before (overfitted). As a result, such models perform very well on training data but has high error rates on test data.

Question 19

How can you deal with overfitting due to low bias?

Answer 19

Low bias occurs when the model is predicting values close to the actual value. It is mimicking the training dataset. The model has no generalization which means if the model is tested on unseen data, it will give poor results.

Bagging (parallel ensemble technique): randomly create subsets of training data and train same algorithm on each subset in parallel, taking consensus result. The combination of models reduces the variance and makes it more reliable compared to a single model.

Question 20

How can you deal with overfitting due to high variance?

Answer 20

Model with high variance pays a lot of attention to training data and does not generalize on the data which it hasn’t seen before (overfitted).

Regularization techniques: penalise higher model coefficients to lower model complexity. Includes LASSO, Ridge Regression and ElasticNet.

Dimensionality reduction via feature selection

Boosting: iterative ensemble technique that adjusts the weights based on the last classification (assigns higher weight to inaccurate predictions)

Question 21

What is the interpretation of a ROC area under the curve?

Answer 21

Receiver operating characteristics (ROC) shows the trade-off between sensitivity and specificity.

• Sensitivity: it is the probability that the model predicts a positive outcome when the actual value is also positive (precision)
• Specificity: it is the probability that the model predicts a negative outcome when the actual value is also negative.

The curve is plotted using the False positive rate (FP/(TN + FP)) and true positive rate (TP/(TP + FN))

The area under the curve (AUC) shows the model performance. If the area under the ROC curve is 0.5, then our model is completely random. The model with AUC close to 1 is the better model.

Question 22

What are the methods of reducing dimensionality?

Answer 22

For dimensionality reduction, we can use feature selection or feature extraction methods.

Dimensionality reduction will decrease the computational cost of training, decrease storage requirements, and may improve the generalisation performance of the model

Question 23

What is feature selection?

Answer 23

Feature selection is a process of selecting optimal features and dropping irrelevant features. We use Filter, Wrapper, and Embedded methods to analyze feature importance and remove less important features to improve model performance.

Question 24

What is feature extraction?

Answer 24

Feature extraction transforms the space with multiple dimensions into fewer dimensions. No information is lost during the process, and it uses fewer resources to process the data. The most common extraction techniques are Linear discriminant analysis (LDA), Kernel PCA, and Quadratic discriminant analysis.

Question 25

What is the Filter method of feature selection?

Answer 25

A subset of features is selected based on their relationship to the target variable. The selection is not dependent of any machine learning algorithm; instead filter methods measure the “relevance” of the features with the output via statistical tests, such as: Pearson’s Correlation: Linear correlation between two continuous variables LDA (Linear Discriminant Analysis): a supervised linear algorithm that projects the data into a smaller subspace k (k < N-1) while maximising the separation between the classes. More specifically, the model finds linear combinations of the features that achieve maximum separability between the classes and minimum variance within each class. ANOVA (Analysis of Variance): tests whether different input categories have significantly different values for the output variable. χ² (Chi-squared): tests whether the occurrences of a specific feature and a specific class are independent using their frequency distribution. The null hypothesis is that the two variables are independent. However, large values of χ² indicate that the null hypothesis should be rejected. When selecting features, we wish to extract those that are highly dependent on the output.

Question 26

What is the Wrapper method of feature selection?

Answer 26

Selecting features by measuring the importance of a subset of features by actually training a model on it. Greedy search and evaluation criterion: the method at each iteration chooses the locally optimal subset of features. Then, the evaluation criterion (evaluation metric such as p-value of R-squared etc) plays the role of the judge. The combination of features that gives the optimal results, according to the evaluation criterion, will be selected. Four approaches: - Forward selection - Backward elimination - Boruta - Genetic algorithm Computationally intensive.

Question 27

What is the Embedded method of feature selection?

Answer 27

Embedded methods combines the advantageous aspects of both Filter and Wrapper methods. Similar to Wrapper methods, however: - perform feature selection and algorithm training in parallel rather than iteratively based on the evaluation metric (feature selection is integral part of the model) - less subject to overfitting than Wrapper methods Methods include: - LASSO (Least Absolute Shrinkage and Selection Operator) - Feature Importance - Tree-based Methods (e.g. random forest) - Permutation Importance

Question 28

How does LASSO work?

Answer 28

Least Absolute Shrinkage and Selection Operator (LASSO) is a shrinkage method that performs both feature selection AND regularization at the same time. It penalises features likely to cause overfitting. It is Linear Regression with L1 regularization. Regularization is a process that shrinks the coefficients (weights) towards zero. This means that you are penalizing more complex models to avoid overfitting. But how does this translate to feature selection? You may have heard of other regularization techniques like Ridge Regression or Elastic net, but LASSO enables coefficients to be set to 0. If a coefficient is zero then the feature is not taken into consideration, thus, it is in a way discarded - helps with feature selection

Question 29

What is regularisation and what methods can do this?

Answer 29

Goal of regularisation is to help the model generalise Regularization is an umbrella term that constrains/ regularizes or shrinks the coefficient estimates towards zero. In other words, this technique discourages learning a more complex or flexible model (learning too much noise), so as to avoid the risk of overfitting. Used when: - Multicollinearity - To filter out noise from data - To prevent overfitting L2: Ridge Regression - RSS is modified by adding the shrinkage quantity L1: LASSO - differs from ridge regression only in penalizing the high coefficients and shrinking to zero (can use for feature selection) ElasticNet - combines LASSO and Ridge

Question 30

What is RSS?

Answer 30

The fitting procedure of linear regression involves a loss function, known as residual sum of squares or RSS. The coefficients are chosen, such that they minimize this loss function.

Question 31

How do you find thresholds for a classifier?

Answer 31

Usually, the threshold of a classifier is 0.5, but in some cases, we need to fine-tune it to improve accuracy. The 0.5 threshold means that if the probability is equal to or above 0.5, it is spam, and if it is lower, then it is not spam. To find the optimal threshold, we can use Precision-Recall curves and ROC curves, grid search, and by manually changing the value to get a better CV.

Question 32

What is Ensemble learning?

Answer 32

Ensemble learning combines insights of multiple machine learning models to improve the accuracy and performance metrics. Simple methods: - Mean/average: average predictions from multiple high-performing models. - Weighted average: assign different weights to machine learning models based on their performance and then combine them. Advanced ensemble methods: - Bagging: used to minimize variance errors. It randomly creates subsets of training data and trains a model on each subset. The combination of models reduces the variance and makes it more reliable compared to a single model. - Boosting: used to reduce bias errors. It is an iterative ensemble technique that adjusts the weights based on the last classification (sequential models). Boosting algorithms give more weight to observations that the previous model predicted inaccurately.

Question 33

In semi-supervised machine learning, what is pseudo labelling?

Answer 33

Pseudo labelling is a method to generate labelled data 1. Train a model with labelled data. 2. Use the trained model to predict labels for the unlabeled data, which creates pseudo-labeled data. 3. Retrain the model with the pseudo-labeled and labeled data together. This process happens iteratively as the model improves and is able to perform with a greater degree of accuracy.

Question 34

In semi-supervised machine learning, what is Self Training?

Answer 34

Self-training is a variation of pseudo labeling. The difference with self-training is that we accept only the predictions that have a high confidence and we iterate through this process several times. In pseudo-labeling, however, there is no boundary of confidence that must be met for a prediction to be used in a model

Question 35

Which metrics are used for evaluating a regression model?

Answer 35

There are three error metrics that are commonly used for evaluating and reporting the performance of a regression model; they are: Mean Squared Error (MSE) - average of the squared differences between predicted and expected target values in a dataset Root Mean Squared Error (RMSE) - units are the same as the original units of the target value that is being predicted (unlike MSE) Mean Absolute Error (MAE) - units match target value units, and changes in MAE are linear and therefore intuitive (MSE and RMSE punish larger errors more than smaller errors, inflating or magnifying the mean error score)

Question 36

What is cross-validation and why is it important?

Answer 36

Assesses the performance and generalization ability of a predictive model. It involves splitting the available dataset into multiple subsets or "folds" to evaluate the model on different combinations of training and validation data. By using cross-validation, you can get a more robust and reliable estimation of a model's performance compared to evaluating it on a single train-test split. It helps to detect overfitting or underfitting issues, assess model stability, and make informed decisions about hyperparameter tuning or model selection.

Question 37

What are some usual applications of Rank algorithms?

Answer 37

1. Search engines 2. Recommender systems 3. Travel agencies (finding best rooms etc)

Question 38

What do ranking algorithms try to predict?

Answer 38

Ranking models typically work by predicting a relevance score s = f(x) for each input x = (q, d) where q is a query and d is a document.

Question 39

What are the two approaches to ranking algorithms?

Answer 39

1. Vector Space Models LM vector embeddings for each query and document, then compute the relevance score f(x) = f(q, d) as the cosine similarity between the vectors embeddings of q and d. 2. Learning to Rank A Machine Learning model that learns to predict a score s given an input x = (q, d) during a training phase where some sort of ranking loss is minimized.

Question 40

What are the six key evaluation metrics for ranking models?

Answer 40

For binary relevance: 1. Mean Average Precision (MAP) 2. Hit Ratio For graded relevance: 3. Discounted Cumulative Gain (DCG) 4. Mean Reciprocal Rank 5. Precision@K 6. Recall@K

Question 41

How does Mean Average Precision work (ranking algorithm)?

Answer 41

Used for tasks with binary relevance, i.e. when the true score y of a document d can be only 0 (non relevant) or 1 (relevant). For a given query q and corresponding documents D = {d₁, …, dₙ}, we check how many of the top k retrieved documents are relevant (y=1) or not (y=0)., in order to compute precision Pₖ and recall Rₖ. For k = 1…n we get different Pₖ and Rₖ values that define the precision-recall curve: the area under this curve is the Average Precision (AP). Finally, by computing the average of AP values for a set of m queries, we obtain the Mean Average Precision (MAP).

Question 42

How does Discounted Cumulative Gain work (ranking algorithm)?

Answer 42

Used for tasks with graded relevance, i.e. typical scale is 0 (bad), 1 (fair), 2 (good), 3 (excellent), 4 (perfect). For a given query q and corresponding documents D = {d₁, …, dₙ}, we consider the the k-th top retrieved document. Gain Gₖ = 2^yₖ – 1 measures how useful is this document Discount Dₖ = 1/log(k+1) penalizes documents that are retrieved with a lower rank. Discounted Gain GₖDₖ for k = 1…n (each doc) The sum of the Discounted Gain is the Discounted Cumulative Gain (DCG). Normalized DCG= DCG/Ideal DCG (ideal scoreif we ranked documents by the true value yₖ) Finally, we usually compute the average of DCG or NDCG values for a set of m queries to obtain a mean value.

Question 43

What types of model are used for ranking?

Answer 43

The base Machine Learning model is usually Decision Tree or Neural Network to compute s = f(x). The choice of loss function is the distinctive element for Learning to Rank models. In general, we have 3 approaches, depending on how the loss is computed: 1. Pointwise Methods 2. Pairwise Methods 3. Listwise Methods

Question 44

What are Pointwise Method loss functions in ranking algorithms?

Answer 44

The total loss is computed as the sum of loss terms defined on each document dᵢ (hence pointwise) as the distance between the predicted score sᵢ and the ground truth yᵢ, for i=1…n. By doing this, we transform our task into a regression problem, where we train a model to predict y. This is the simplest loss to implement, and is used with regression metrics such a MSE, MAE etc Problem is that we need true relevant scores to train the model. In most cases, we might only know the ordering of items and not the absolute relevant scores.

Question 45

What are Pairwise Method loss functions in ranking algorithms?

Answer 45

The total loss is computed as the sum of loss terms defined on each pair of documents dᵢ, dⱼ (hence pairwise) , for i, j=1…n. The objective on which the model is trained is to predict whether yᵢ > yⱼ or not, i.e. which of two documents is more relevant. By doing this, we transform our task into a binary classification problem. This only works with relative preference: given two documents, we want to predict if the first is more relevant than the second, and not with absolute relevance. Binary classification evaluation metrics can be used for the loss, e.g. Binary Cross Entropy loss, or you can use Gradient Descent directly (no loss needed - only Gradients)

Question 46

What are Listwise Method loss functions in ranking algorithms?

Answer 46

The loss is directly computed on the whole list of documents (hence listwise) with corresponding predicted ranks. In this way, ranking metrics such as DCG can be more directly incorporated into the loss. In contrast to pointwise/pairwise, solves the problem more directly by maximizing the evaluation metric - state of the art results achieved with this approach eg. LambdaLoss RankNet, LambdaRank, SoftRank, ListNet

Question 47

What is Cross-Entropy loss?

Answer 47

Cross-Entropy loss is a most important cost function. It is used to optimize classification models. Cross-Entropy takes the output probabilities (P) from SoftMax and measures the distance from the truth values. Also called logarithmic loss, log loss or logistic loss. Each predicted class probability is compared to the actual class desired output 0 or 1 and a score/loss is calculated that penalizes the probability based on how far it is from the actual expected value. The penalty is logarithmic in nature yielding a large score for large differences close to 1 and small score for small differences tending to 0. Cross-entropy loss is used when adjusting model weights during training. The aim is to minimize the loss, i.e, the smaller the loss the better the model. A perfect model has a cross-entropy loss of 0.

Question 48

What is Entropy in ML?

Answer 48

A key measure in information theory Entropy of a random variable X is the level of uncertainty inherent in the variables possible outcome. The greater the value of entropy,H(x) , the greater the uncertainty for probability distribution and the smaller the value the less the uncertainty. Entropy is between 0 and 1

Question 49

What is the difference between Categorical Cross-Entropy and Sparse Categorical Cross-Entropy?

Answer 49

Both have the same cross entropy loss function. The only difference is how truth labels are defined. - Categorical cross-entropy is used when true labels are one-hot encoded, for example, we have the following true values for 3-class classification problem [1,0,0], [0,1,0] and [0,0,1]. - Sparse categorical cross-entropy uses truth labels that are integer encoded, for example, [1], [2] and [3] for 3-class problem.

Question 50

In Discounted Cumulative Gain, what is the Gain?

Answer 50

Gk = 2^yk -1 Measures how useful is this document

Question 51

In Discounted Cumulative Gain, what is the Discount?

Answer 51

Dk = 1 / log(k+1) Penalises documents that are retrieved with a lower rank

Question 52

What is Gradient Descent?

Answer 52

Gradient descent is a first-order iterative optimization algorithm for finding a local minimum of a differentiable function. The idea is to take repeated steps in the opposite direction of the gradient (or approximate gradient) of the function at the current point, because this is the direction of steepest descent.

Question 53

What is least-squares regression?

Answer 53

Where the ML goal is to "teach" a model, F, to predict values of the form y = F(x) by minimizing the mean squared error

Question 54

What is Gradient Boosting?

Answer 54

Variant of ensemble methods where you create multiple weak models and combine them to get better performance as a whole. Specific to regression problems with squared loss (MSE etc) Qualities: - Powerful enough to find any nonlinear relationship between your model target and features - Can deal with missing values, outliers, and high cardinality categorical values without any special treatment. e.g. XGBoost or LightGBM

Question 55

What are residuals?

Answer 55

prediction errors

Question 56

Define information retrieval

Answer 56

Information retrieval is the process of retrieving relevant information from a collection of unstructured or semi-structured data, typically text documents, in response to user queries or information needs. It involves techniques and algorithms to efficiently search, analyze, and present information to users.

Question 57

What is 'relevance' with regard to information retrieval?

Answer 57

In the context of information retrieval, "relevance" refers to the degree to which a document or information item satisfies the information needs or requirements of a user. It is a measure of how closely a document aligns with the user's query or information-seeking intent.

Question 58

What are the key components of an information retrieval system?

Answer 58

1. Document Collection 2. Indexing: structured representation of the document collection to facilitate efficient searching 3. Query Processing: parsing and understanding the user's query and formulating an appropriate search strategy. 4. Retrieval Model: mathematical or statistical framework used to assess the relevance of documents to a given query- determines how documents are scored or ranked based on their similarity or match to the query 5. Ranking Algorithm: sorts retrieved documents based on their relevance scores 6. User Interface: provides the means for users to interact with the information retrieval system 7. Evaluation Metrics: Common metrics include precision, recall, F1 score, mean average precision (MAP), and normalized discounted cumulative gain (NDCG). 8. Relevance Feedback: feedback on the retrieved results

Question 59

How does Hit Ratio work in Information Retrieval?

Answer 59

It is simply the fraction of queries for which the correct answer is included in the recommendation list of length L.

Question 60

How does Mean Reciprocal Rank work in Information Retrieval?

Answer 60

Measures how far down the ranking the first relevant document is. If MRR is close to 1, it means relevant results are close to the top of search results - what we want! Lower MRRs indicate poorer search quality, with the right answer farther down in the search results.

Question 61

How do precision@K and recall@K work in Information Retrieval?

Answer 61

In the top K documents, what is the precision/recall i.e. precision - proportion of top k that are TP (indeed relevant) recall - proportion of all positives (relevant documents) that are in top k

Question 62

What are LambdaRank and LambdaMart?

Answer 62

These are ranking algorithms, which are in-between pairwise and listwise methods. LambdaRank: based on gradient boosting decision tree models LambdaRank: based on neural networks

Question 63

What are the different types of Relevance feedback?

Answer 63

1. Explicit - assessors indicate relevance of results explicitly using a binary or graded relevance system 2. Implicit - based on user behaviours (documents selected for view) 3. Pseudo - - Take the top 10-50 results returned by initial query - Select top 20-30 terms from these documents using e.g. tf-idf weights. - Do Query Expansion; add these terms to query, then match the returned documents for this query and finally return the most relevant documents

Question 64

What is a Web Crawler?

Answer 64

A web crawler is a computer program that crawls through the web in a predefined and methodical manner to collect data. The web crawler tool pulls together details about each webpage: titles, images, keywords, other linked pages, etc. It automatically maps the web to search documents, websites, RSS feeds, and email addresses. It then stores and indexes this data. Web crawlers use several algorithms to rate the value of the content or the quality of the links in its index. These rules determine its crawling behavior: which sites to crawl, how often to re-crawl a page, how many pages on a site to be indexed, and so on. When it visits a new website, it downloads its robots.txt file—the “robots exclusion standard” protocol designed to restrict unlimited access by web crawler tools. The file contains information of sitemaps (the URLs to crawl) and the search rules (which of the pages are to be crawled and which parts to ignore).

Question 65

What is Query Expanision in Information Retrieval?

Answer 65

Query expansion is the process of reformulating a given query to improve retrieval performance in information retrieval operations, particularly in the context of query understanding. Query expansion involves techniques such as: - Finding synonyms of words, and searching for the synonyms as well - Finding semantically related words (e.g. antonyms, meronyms, hyponyms, hypernyms) - Finding all the various morphological forms of words by stemming each word in the search query - Fixing spelling errors and automatically searching for the corrected form or suggesting it in the results - Re-weighting the terms in the original query

Question 66

What is Data Leakage?

Answer 66

Data leakage occurs when information from the validation set inadvertently leaks into the training process. It can happen if the validation set is used to inform decisions during model training, such as feature selection or hyperparameter tuning. In such cases, the model's performance on the validation set can provide a more reliable evaluation of its generalization ability.

Question 67

What are the assumptions of linear regression?

Answer 67

1. Linearity: relationship between predictor variables and response variable is assumed to be linear 2. Independence: observations are independent of each other (no correlation or dependence) 3. Homoscedasticity: variability of the residuals (differences between observed and predicted values) is constant across all levels of the predictor variables 4. Normality: residuals follow a normal distribution. This allows for estimation of confidence intervals and hypothesis tests in linear regression. **It is not necessary for the predictor variables to follow a normal distribution.** 5. No Multicollinearity: Multicollinearity refers to a high correlation between predictor variables No Endogeneity: predictor variables are exogenous, meaning they are not affected by the response variable. Endogeneity can lead to biased and inconsistent coefficient estimates.

Question 68

What are the four main types of linear model?

Answer 68

1. Linear regression 2. Logistic regression 3. Ridge regression 4. LASSO regression

Question 69

What are the five main types of tree-based model?

Answer 69

1. Decision trees 2. Random forest 3. Gradient boosting regression 4. XG Boost 5. Light GBM Regressor

Question 70

What are the three main types of clustering model?

Answer 70

1. K-means 2. Hierarchical clustering 3. Gaussian mixture model

Question 71

How does K-means clustering work?

Answer 71

1. Number of centroids defined (use elbow method to find optimum) 2. Randomly assign centroids 3. Find data points closest to each centroid (Euclidean distance) 4. Iterate to reduce Euclidean distance between centroid and cluster data

Question 72

When is one hot encoding used and what are its limitations?

Answer 72

One hot encoding is used when you need to convert categorical data into numeric. When there are too many categories to encode, multicollinearity is a risk Use pandas .get_dummies() Most algos require one hot as they use numerical data Algorithms that do not require an encoding are algorithms that can directly deal with joint discrete distributions such as Markov chain / Naive Bayes / Bayesian network, tree based

Question 73

What is Principle Component Analysis?

Answer 73

PCA statistics is the science of analyzing all the dimensions on a dataset and reducing them as much as possible while preserving the exact information. When to use? - Dimensionality reduction - Categorize the dependent and independent variables in your data - Eliminate noise components in your dimension analysis Steps: - Standardise data - Compute covariance matrix to detect correlations - Compute eigenvectors and eigenvalues from covariance matrix to identify Principle Components - Create feature vector to define Principle Components - Recast data along Principle Components axis

Question 74

What is meant by Data Standardisation?

Answer 74

1. The range of variables in a dataset is standardized to analyze the contribution of each variable equally. 2. Calculating the initial variables will help categorize the variables that are dominating the other variables of small ranges. 3. This will help you attain biased results at the end of the analysis. 4. To transform the variables of the same standard, you can follow the following formula: Z = (value - mean) / st dev

Question 75

What are the key steps involved with a decision tree?

Answer 75

1. Take the entire data set as input (accepts numerical and categorical data) 2. Calculate entropy of the target variable, as well as the predictor attributes 3. Calculate information gain of all attributes (we gain information on sorting different objects from each other) 4. Choose attribute with the highest information gain as the root node 5. Repeat procedure on every branch until the decision node of each branch is finalized 6. Take average or consensus of nodes to infer tree prediction

Question 76

In machine learning, what is information gain and when is it used?

Answer 76

Information gain is a measure of how much information a feature provides about a class: it measures how uncertainty in the target variable is reduced, given a set of independent variables. Information gain helps to determine the order of attributes in the nodes of a decision tree. Defined by: Gain = Entropy(parent node) - Entropy(child node) Information gain is the entropy we “lost” after splitting data at a node (how good was this split) Want to decrease entropy as progress through decision tree

Question 77

What are the key differences between decision trees and random forest?

Answer 77

Main difference: establishing root nodes and segregating nodes is done randomly in Random Forest. - RF = ensemble of decision tress - RF employs the bagging method to generate the required prediction - RF more time and computation intensive - RF less sensitive to overfitting

Question 78

Differentiate between univariate, bivariate, and multivariate analysis.

Answer 78

Univariate data contains only one variable. The purpose of the univariate analysis is to describe the data and find patterns that exist within it. Bivariate data involves two different variables. The analysis of this type of data deals with causes and relationships and the analysis is done to determine the relationship between the two variables. Multivariate data involves three or more variables. It is similar to a bivariate but contains more than one dependent variable.

Question 79

How do you calculate Mean Square Error and RMSE?

Answer 79

Measures error by using the average squared difference between observed and predicted values. MSE = SumOf(yi - y(hat)i) squared / n RMSE is square root of this

Question 80

What is the elbow method?

Answer 80

Used to define optimal number of clusters for k-means Determine the k-value by iteratively clustering k=1 to k=n (Here n is the hyperparameter that we choose as per our requirement). For every value of k, we calculate the within-cluster sum of squares (WCSS) value. WCSS - the sum of square distances between the centroids and each point in cluster. To determine optimal number of clusters(k), plot a graph of k versus WCSS value

Question 81

Which ML algorithm can be used to impute both categorical and numerical missing data?

Answer 81

K Nearest Neighbours

Question 82

What are True Positive Rate and False Positive Rate?

Answer 82

TPR defines the probability that an actual positive will turn out to be positive. TPR=TP/TP+FN FPR defines the probability that an actual negative result will be shown as a positive one i.e the probability that a model will generate a false alarm. FPR=FP/TN+FP These are used to plot the ROC curve (y=TPR, x=FPR)

Question 83

What is decision tree pruning?

Answer 83

Pruning simplifies the decision tree by reducing the rules; this helps to avoid complexity, reduce overfitting and improves accuracy. Simplifies a decision tree by removing the weakest rules. Pruning is often distinguished into: - Pre-pruning (early stopping) stops the tree before it has completed classifying the training set - Post-pruning allows the tree to classify the training set perfectly and then prunes the tree Post-pruning starts with an unpruned tree, takes a sequence of subtrees (pruned trees), and picks the best one through cross-validation.

Question 84

What is the difference between an error and a residual error?

Answer 84

Error = difference between prediction and actual value Residual = difference between arithmetic mean of a group of values and the observed group of values

Question 85

What is overfitting?

Answer 85

- Occurs when the learning power of the model is too high OR the data is too small - Model learns the noise rather than the information - Model performs badly on unseen data Will have overfitting problem if have regression model and number of data points < number of features Can remedy by: - reducing learning power of - Increasing size of training data - Regularisation

Question 86

What are L1 and L2 regularisation?

Answer 86

You can have Ln regularisations, but L1 (Lasso) and L2 (ridge) are most common ElasticNet is a hybrid of these two All features must be on comparable scales for regularisation to occur Both can be applied to all parametric models (regression, SVM, neural networks) L1 is binary - adds weights of zero or 1, so cannot be used for feature selection For correlated features, L1 selects the best on whereas L2 spreads the error amongst both

Question 87

In regularisation, what is alpha

Answer 87

Alpha controls the amount of feature shrinkage - the larger alpha, the greater the shrinkage

Question 88

What is ridge regression?

Answer 88

L2 Form of regularisation L2 shrinks parameters and reduces influence of unimportant features. It is more stable than L1. It is differentiable, so gradient descent can be used to optimise. Spreads error amongst all terms; does not shrink parameters to zero, so cannot be used for feature selection

Question 89

What Problems Do Multicollinearity Cause?

Answer 89

Multicollinearity causes the following two basic types of problems: The coefficient estimates can swing wildly based on which other independent variables are in the model. The coefficients become very sensitive to small changes in the model. Multicollinearity reduces the precision of the estimated coefficients, which weakens the statistical power of your regression model. You might not be able to trust the p-values to identify independent variables that are statistically significant.