Precision and recall

From NoskeWiki
Jump to: navigation, search

About

I always forget which is which for precision and recall in Machine Learning / estimation.

Precision versus Recall

Imagine a machine learning algorithm to detect cat photos.

  • Precision -> of the number you thought were positive.... what fraction are actually positive. (think: what fraction of darts hit targets)
    • True Positives / (True Positives + False Positives)
  • Recall -> what fraction of cats were predicted as cats? (think: how many targets were hit)
    • True Positives / (True Positives + False Negatives)

If you just blindly say everything is a cat we get 100% recall, but really low precision (a tonne of non-cat photos we said were cat). Machine learning papers often show a "precision recall curve" which shows that as you adjust a threshold to guess more "cats".... if we only pick 99.9% confidence of a cat, it's probably almost always right (100% recision), but terrible recall. But as we go down to 80% sure of a cat, we get better recall (we capture a higher fraction of cats), but the trade off is that we're probably selecting more and more non-cats.

Said Another Way

Your boss asks:

  • What percent of your predictions were correct? "accuracy"
  • What percent of the positive cases did you catch? "recall"
  • What percent of positive predictions were correct? "precision"

Table of Confusion

A table of confusion (sometimes also called a confusion matrix), is a table with two rows and two columns that reports the number of false positives, false negatives, true positives, and true negatives.

True positives

(cats
correctly classified as cats)
False negatives

(cats
incorrectly classified as dogs)
False positives

(non-cat
incorrectly classified as cats)
True negatives

(non-cat,
correctly classified as non-cats)

Accuracy

Accuracy: The fraction of predictions we got right...


Links