mlresearch.metrics.ALScorer

class mlresearch.metrics.ALScorer(score_func, sign=1, **kwargs)

Make an Active Learning scorer from a AL-specific metric or loss function.

This factory class wraps scoring functions to be used in ModelSearchCV and GridSearchCV. It takes a score function, such as area_under_learning_curve() or data_utilization_rate() and is used to score an AL simulation. The signature of the call is (estimator, X, y) where estimator is the model to be evaluated, X is the data and y is the ground truth labeling (or None in the case of unsupervised models).

Parameters:

score_funccallable

Score function (or loss function) with signature score_func(y, y_pred, **kwargs).

signint, default=1

Use 1 to keep the original variable’s scale, use -1 to reverse the scale.

Returns:

scorercallable

Callable object that returns a scalar score.

get_metadata_routing()

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:

routingMetadataRequest

A MetadataRequest encapsulating routing information.

set_score_request()

Placeholder to overwrite sklearn’s _BaseScorer.set_score_request function. It is not used and was raising a docstring error with scikit-learn v1.3.0.