# Cost Functions, Performance Measures and Optimization Flashcards

What is the Cost Function?

The cost function is also called the loss function.

It is the difference between the predicted values and the ground truth, y.

What is the cost function used for regression?

We call it RMSE. Root-Mean-Square-Error

What is the cost function that we use to classify?

We have used a modified RMSE that is modified so that it returns either 0 or 1 by multiplying by the ground truth

What is regularization?

Regularization is something we use to prevent overfitting in our model.

We add some information to the cost function to change the algorithm

What is overfitting?

Overfitting is when the learning rate is too high so that the algorithm will over adjust when near the best fitting vector.

It could also describe when the model has been trained with too much training sets so that it learns the noise and errors of the data as features.

With classifiers, usually, how many nodes should there be in each layer as we go towards the output layer?

There should be fewer and fewer nodes.

e.g if there are 10 input nodes there should be fewer nods in layer 1 and so on.

Which parameters affect the learning

number of hidden layers, number of nodes in those layers, number of training iterations. Biasses

What is a hyperparameter?

A hyperparameter is the parameters that affect learning, but now how it learns. So.. learning rate and regularization parameters.

What are the performance measures for regression models?

RMSE or Mean Absolute error.

What is the performance measures for classification

Accuracy

F1 scores

Youden’s Index

precision

What is the accuracy performance measure for classification models?

It measures how accurate your model is at predicting and classifying correctly.

“The proportion of true results among the total number of cases”

What is the issue with accuracy performance measure?

It might not be useful.

If we predict how many asteroids we can just say zero and be right 99 % of the time. But the model is not very useful then.

which activation functions would you use for classification?

sigmoid or tanh

When is accuracy a good performance measure to use?

When we want to classify almost evenly distributed categories.

It does not make sense to use in very imbalanced cases.

What is the precision performance measure?

It is how often your model predicts truly correct.

How often your model predicts a case correctly