TrainedLearner.test

TrainedLearner.test(performanceFunction, testX, testY=None, arguments=None, *, useLog=None, **kwarguments)

Evaluate the performance of the trained learner.

Evaluation of predictions of testX using the argument performanceFunction to do the evaluation. Equivalent to having called trainAndTest, as long as the data and parameter setup for training was the same.

Parameters:

  • performanceFunction (function) – The function used to determine the performance of the learner. Pre-made functions are available in nimble.calculate. If hyperparameter tuning and the Tuning instance does not have a set performanceFunction, it will utilize this function as well.

  • testX (nimble.core.data.Base) – The object containing the test data.

  • testY (identifier, nimble Base object) –

    • identifier - A name or index of the feature in testX containing the labels.

    • nimble Base object - contains the labels that correspond to testX.

  • performanceFunction – If cross validation is triggered to select from the given argument set, then this function will be used to generate a performance score for the run. Function is of the form: def func(knownValues, predictedValues). Look in nimble.calculate for pre-made options. Default is None, since if there is no parameter selection to be done, it is not used.

  • arguments (dict) – Mapping argument names (strings) to their values, to be used during training and application. eg. {‘dimensions’:5, ‘k’:5} To make use of multiple permutations, specify different values for a parameter as a tuple. eg. {‘k’: (1,3,5)} will generate an error score for the learner when the learner was passed all three values of k, separately. These will be merged with kwarguments for the learner.

  • useLog (bool, None) – Local control for whether to send results/timing to the logger. If None (default), use the value as specified in the “logger” “enabledByDefault” configuration option. If True, send to the logger regardless of the global option. If False, do NOT send to the logger, regardless of the global option.

  • kwarguments – Keyword arguments specified variables that are passed to the learner. To make use of multiple permutations, specify different values for parameters as a tuple. eg. arg1=(1,2,3), arg2=(4,5,6) which correspond to permutations/argument states with one element from arg1 and one element from arg2, such that an example generated permutation/argument state would be arg1=2, arg2=4. Will be merged with arguments.

Returns:

performance – The calculated value of the performanceFunction after the test.

See also

nimble.trainAndTest, apply

Examples

>>> from nimble.calculate import fractionIncorrect
>>> lstTrain = [[1, 0, 0, 1],
...             [0, 1, 0, 2],
...             [0, 0, 1, 3],
...             [1, 0, 0, 1],
...             [0, 1, 0, 2],
...             [0, 0, 1, 3]]
>>> lstTest = [[1, 0, 0, 1], [0, 1, 0, 2], [0, 0, 1, 3]]
>>> ftNames = ['a', 'b', 'c', 'label']
>>> trainData = nimble.data(lstTrain, featureNames=ftNames)
>>> testData = nimble.data(lstTest, featureNames=ftNames)
>>> knn = nimble.train('nimble.KNNClassifier', trainX=trainData,
...                    trainY='label')
>>> knn.test(fractionIncorrect, testX=testData, testY='label')
0.0