L4: Foundation of ML and Linear Regression Flashcards
Which of the following statements hold true for supervised ML? (Select all correct)
A) the algorithm is trained on a labeled dataset, where the input data is associated with corresponding output labels
B) The goal is for the algorithm to learn a mapping or relationship between the input features and the output labels
C) logistic regression is a form of supervised learning
D) decision trees is a form of supervised learning
E) linear regression is a form of supervised leaning
In supervised learning, all hold true
A) the algorithm is trained on a labeled dataset, where the input data is associated with corresponding output labels
B) The goal is for the algorithm to learn a mapping or relationship between the input features and the output labels
C) logistic regression is a form of supervised learning
D) decision trees is a form of supervised learning
E) linear regression is a form of supervised leaning
In the case of logistic regression, supervised learning is used for binary classification problems, where the output variable is categorical (e.g., 0 or 1). The LR model estimates the probability that a given input belongs to a particular class and makes predictions based on these probabilities.
TRUE/ FALSE
TRUE
In the context of decision trees, the algorithm learns (with supervised learning) a set of hierarchical decision rules based on the features of the data to make predictions about the output label.
TRUE/ FALSE
TRUE
Unsupervised learning involves NOT…
Select all wrong statements:
A) training a model on an labelled dataset
B) the algorithm explores the inherent structure or patterns within the data without explicit guidance
C) Common tasks include clustering, dimensionality reduction, and density estimation
D) The goal is to discover relationships or groupings in the data without predefined output labels
Wrong:
A) unsupervised learning involves training a model on an labelled dataset
the model is trained on an un-labelled dataset, in contrary to supervised dataset
Which of the following statements are true for reinforcement learning?
A) a learning agent evolves behaviors to solve tasks using evaluative feedback
B) the agent is punished or rewarded as consequence of its actions
C) it requires interactions between agent and environment
D) the agent learns through experience (trial/error)
All statements are true:
A) a learning agent evolves behaviors to solve tasks using evaluative feedback
B) the agent is punished or rewarded as consequence of its actions
C) it requires interactions between agent and environment
D) the agent learns through experience (trial/error)
In linear regression, which measure tells us how good the prediction is?
Mean squarred error (MSE) is used as the loss function in linear regression, telling us how good the prediction is
What are the limitations of ML?
Example/ hint: if the temperature measurer breaks down for one day, you must find a way to take this into account
There are many areas where things can go wrong in ML, leading the machine to learn something wrong.
When using ML to distinguish between blueberry muffins and chihuahuas, the “chihuahua” label is stratistically associatted with big eyes, small bodies, pointy ears, etc.
What are the potential limitations on ML in this context?
The ML algorithm sees the world as a set of pixel values. But muffins and chihuahuas have similar pixel values…
The same issue might also be present when distinguishing between chihuahuas and other dog breeds with similar attributes.
Limitation: it can be wrong
When using ML to distinguish between blueberry muffins and chihuahuas, it poses limitations due to the attributes of muffins and chihuahuas being similar. What is a solution to this?
You want to define features that specifically discriminate and thus separate between doubt objects:
- distance between eyes
- max three dots
- max x kilo
- smell and taste data
Once a rule/ prediction is established, re-testing and re-training is necessary with new data to account for:
A) monoticity
B) changes in rules
C) veloroticity
B) changes in rules
I.e., when the future becomes different from past, retraining and retesting is necessary to maintain good predictive performance on future data
What is the cross-validation trade-off?
Cross-validation trade-off: if you use part of the data to test the rule made based on the remainder of the data, this allows the model to be tested out. However, this also means that less data is used for building the model
What is cross-validation?
Cross validation entails splitting your data into one part for training and the held-out set for testing.
What is the goal of cross-validation?
To avoid or mitigate overfitting
Which of the following statements are true about bias in the bias-variance tradeoff in ML?
A) Bias refers to the error introduced by approximating a real-world problem with a simplified model
B) High bias models may oversimplify the underlying patterns in the data and lead to systematic errors.
C) Low bias is often associated with underfitting
D) Bias is the error introduced by using a complex model that is highly responsive to the training data
A) Bias refers to the error introduced by approximating a real-world problem with a simplified model
B) High bias models may oversimplify the underlying patterns in the data and lead to systematic errors.
WRONG:
C) Low bias is often associated with underfitting –> HIGH bias has this problem
D) Bias is the error introduced by using a complex model that is highly responsive to the training data –> this is the case for high variance
In the bias-variance tradeoff in ML, which of the following statements are FALSE about variance? (Select all wrong answers)
A) variance is the error introduce by using a complex model highly responsive to the training data
B) high variance models may capture noise or random fluctuations in the training data
C) high variance leads to poor generalisation on new, unseen data
D) High variance is often associated with overfitting
All options are true
A) variance is the error introduce by using a complex model highly responsive to the training data
B) high variance models may capture noise or random fluctuations in the training data
C) high variance leads to poor generalisation on new, unseen data
D) High variance is often associated with overfitting
Simpler models tend to have higher variance but lower bias – and vice-versa for complex models. But very complex models can result in overfitting.
TRUE/ FALSE?
FALSE. Actually true statement:
Simpler models tend to have higher BIAS but lower VARIANCE – and vice-versa for complex models. But very complex models can result in overfitting
Predictive modeling works by leveraging correlations between feature values (input) and output. Ideally, our predictors would each contribute independent sources of information about the outcome, and this is often the case
TRUE/FALSE
FALSE
Instead:
Predictive modeling works by leveraging correlations between feature values (input) and output. Ideally, our predictors would each contribute independent sources of information about the outcome, but this often not the case
Which of the following statements are NOT true about co-linearity?
A) it occurs when two+ features are highly correlated with one another.
B) it can cause problems if doing statistical inference because if predictors increase or decrease together it can be hard to determine their separate effects the output
C) Generally the solution is to remove one of the redundant predictors
D) Since we are more interested in prediction and not in explanatory modeling, it is even more important to take into account
WRONG:
D) Since we are more interested in prediction and not in explanatory modeling, it is even more important to take into account
Instead: Since we are more interested in prediction and not in explanatory modeling, we won’t worry too much about the issue of (multi)co-linearity.
You can make a correlation matrix to explore correlations between variables that are both numeric and categorical
TRUE/FALSE
FALSE
Correlation matrix, or correlation in general, can only be mapped for numeric variables, since it is not possible to see this relation for variables with a finite number of outcomes (e.g. factors)
In linear regression, the intercept term alone tells you the expected response variable output if explanatory variables had a value of 0
TRUE/FALSE
TRUE
