nimble.calculate.confusionMatrix

nimble.calculate.confusionMatrix(knownValues, predictedValues, labels=None, convertCountsToFractions=False)

Generate a confusion matrix for known and predicted label values.

The confusion matrix contains the counts of observations that occurred for each known/predicted pair. Features represent the known classes and points represent the predicted classes. Optionally, these can be output as fractions instead of counts.

Parameters:

• knownValues (nimble Base object) – The ground truth labels collected for some data.

• predictedValues (nimble Base object) – The labels predicted for the same data.

• labels (dict, list) – As a dictionary, a mapping of from the value in knownLabels to a more specific label. A list may also be used provided the values in knownLabels represent an index to each value in the list. The labels will be used to create the featureNames and pointNames with the prefixes known_ and predicted_, respectively. If labels is None, the prefixes will be applied directly to the unique values found in knownLabels.

• convertCountsToFractions (bool) – If False, the default, elements are counts. If True, the counts are converted to fractions by dividing by the total number of observations.

Returns:

Base – A confusion matrix nimble object matching the type of knownValues.

Notes

Metrics for binary classification based on a confusion matrix, like truePositive, recall, precision, etc., can also be found in the nimble.calculate module.

Examples

Confusion matrix with and without alternate labels.

>>> known = [[0], [1], [2],
...          [0], [1], [2],
...          [0], [1], [2],
...          [0], [1], [2]]
>>> pred = [[0], [1], [2],
...         [0], [1], [2],
...         [0], [1], [2],
...         [1], [0], [2]]
>>> knownObj = nimble.data(known)
>>> predObj = nimble.data(pred)
>>> cm = confusionMatrix(knownObj, predObj)
>>> cm
<Matrix 3pt x 3ft
               known_0  known_1  known_2
             ┌──────────────────────────
 predicted_0 │    3        1        0
 predicted_1 │    1        3        0
 predicted_2 │    0        0        4
>
>>> labels = {0: 'cat', 1: 'dog', 2: 'fish'}
>>> cm = confusionMatrix(knownObj, predObj, labels=labels)
>>> cm
<Matrix 3pt x 3ft
                  known_cat  known_dog  known_fish
                ┌─────────────────────────────────
  predicted_cat │     3          1          0
  predicted_dog │     1          3          0
 predicted_fish │     0          0          4
>

Label objects can have string values and here we output fractions.

>>> known = [['cat'], ['dog'], ['fish'],
...          ['cat'], ['dog'], ['fish'],
...          ['cat'], ['dog'], ['fish'],
...          ['cat'], ['dog'], ['fish']]
>>> pred = [['cat'], ['dog'], ['fish'],
...         ['cat'], ['dog'], ['fish'],
...         ['cat'], ['dog'], ['fish'],
...         ['dog'], ['cat'], ['fish']]
>>> knownObj = nimble.data(known)
>>> predObj = nimble.data(pred)
>>> cm = confusionMatrix(knownObj, predObj,
...                      convertCountsToFractions=True)
>>> cm
<DataFrame 3pt x 3ft
                  known_cat  known_dog  known_fish
                ┌─────────────────────────────────
  predicted_cat │   0.250      0.083      0.000
  predicted_dog │   0.083      0.250      0.000
 predicted_fish │   0.000      0.000      0.333
>