Exercise 2 - ML part 2

Question 1

what does the model M do?

Answer 1

encode, stores and retrieves the outcomes of a learning process

Question 2

What does H do?

Answer 2

The hypothesis space determines which aspects of the data are captured and how they are represented

Question 3

How is it called when you combine hypothesis?

Answer 3

Ensemble method

Question 4

What is boosting

Answer 4

computes a strong learner by incrementally constructing an ensemble of hypothesis

Question 5

law of parsimony that is stated in the principle of Occam’s razor

Answer 5

Of two competing theories, the simpler explanation of an entity is to be preferred

Question 6

discriminate data

Answer 6

compute prediction y for an input x

Question 7

What are discriminative models based on?

Answer 7

posterior probabilities P(y|x)

Question 8

What are generative models based on?

Answer 8

prior probabilities P(x|y)

• -> How likely is it to have data of a certain label?
• -> Can be computed via the Bayes’ algorithm
• -> Generative models are compact representations that have considerably less parameters

Question 9

How can overfitting be detected?

Answer 9

By applying h to unseen data samples

Question 10

Avoidance of overfitting through

Answer 10

• regularization
• more training data (increasing the complexity of the dataset)
• dataset augmentation (e.g. adding noise)

Question 11

value domain of learning methods

Answer 11

• discrete (classification)

- continuous (regression)

Question 12

different models

Answer 12

deterministic - stochastic
parametric - nonparametric
generative discriminative

Question 13

nonparametric

Answer 13

An easy to understand nonparametric model is the k-nearest neighbors algorithm that makes predictions based on the k most similar training patterns for a new data instance. The method does not assume anything about the form of the mapping function other than patterns that are close are likely to have a similar output variable.

Question 14

parametric

Answer 14

e. g. linear function

- you make assumptions

Question 15

Types of reasoning

Answer 15

• inductive
• deductive
• transductive

Question 16

perceptron

Answer 16

Linear classifier that is based on a single Neuron with a digital threshold function. It outputs +1 (if x >=0) and -1 (if x < 0).
- the perceptron criterion punishes only incorrectly classified samples

Question 17

Can the perceptron learning rule find a solution?

Answer 17

If a solution exists, so if the data set is linearly separable, then the perceptron learning algorithm finds a solution within a finite number of steps
- the solution depends on the initialization of the parameters and the order of presentation of the training samples

Question 18

What led to the AI winter?

Answer 18

• The perceptron learning rule only converges for linearly separable data sets. It therefore cannot classify the XOR dataset correctly. This led to the abandonment of connectionism for almost two decades.

Question 19

Examples of basis functions

Answer 19

• linear functions

- sigmoid functions etc.

Question 20

What is interpolation

Answer 20

a function that maps to every training data instance

Question 21

Is an analog neuron model a generalized linear model?

Answer 21

yes

Question 22

Issues of Least Squares Linear Classification

Answer 22

sensitive to outliers

Question 23

What solves the problem of Least Squares Linear Classification

Answer 23

Support Vector Machines (SVM) minimizes the generalization error.