research.metrics.ALScorer

class research.metrics.ALScorer(score_func, sign=1)[source]

Make an Active Learning scorer from a AL-specific metric or loss function.

This factory class wraps scoring functions for use in ModelSearchCV and GridSearchCV. It takes a score function, such as area_under_learning_curve() or data_utilization_rate() and is used to score an AL simulation. The signature of the call is (estimator, X, y) where estimator is the model to be evaluated, X is the data and y is the ground truth labeling (or None in the case of unsupervised models).

Parameters
score_funccallable

Score function (or loss function) with signature score_func(y, y_pred, **kwargs).

signint, default=1

Use 1 to keep the original variable’s scale, use -1 to reverse the scale.

Returns
scorercallable

Callable object that returns a scalar score.

Methods

__call__(estimator, X, y_true[, sample_weight])

Evaluate predicted target values for X relative to y_true.