Evaluation of Categorisation Algorithms

Question 1

What is the formula for Precision?

Answer 1

Precision = TP / (TP + FP)

All along the R side (the RA side).
Jade was a very precise RA, he’d use a pin to touch the left side of the confusion matrix.

Question 2

In layperson terms, what does Precision measure?

Answer 2

Precision measures the proportion of the Algorithm’s “positive” results that were positive in Reality.

Of all the “positives” determined by the algorithm, what proportion were ‘positive’ in reality?

ie True positives / All algo positives (ie TP + FP)

(AR)
    Algo
      \+     -
R+ tp    fn
   - fp    tn

Question 3

What goes across the top of the confusion matrix?

Answer 3

Algorithm results.

Positive and negative.

Question 4

What goes along the left axis of the confusion matrix?

Answer 4

Reality.

Positive.
Negative.

Question 5

What goes in the 4 boxes of the confusion matrix?

Answer 5

(AR)

Algo
  \+     - R+ tp    fn    - fp    tn

Question 6

What does Recall measure?

Answer 6

Of all Real positives, what proportion were predicted correctly by The Machine?

Recall = TP / TP + FN

Question 7

What is the formula for Recall?

Answer 7

Recall = TP / TP + FN

Question 8

What epidemiological term is the same as the data science term Recall?

Answer 8

Sensitivity

Can you recall how sensitive you used to be before you were married!!

Question 9

What epidoologocal term is the same as the data science term Precision?

Answer 9

Positive Predictive Value (PPV)

Given that people have the disease, what proportion are picked up by the test?