mlresearch.metrics.AlphaPrecision

class mlresearch.metrics.AlphaPrecision(scorer_real, alpha=0.05)[source]

Measures synthetic data fidelity. It estimates the probability that a synthetic sample resides in the $alpha$-support of the real distribution.

This is an implementation of the metric proposed in [1].

Warning

This metric is not listed in the get_scorer_names function since it is following an unconventional structure.

Parameters:
scorer_realfunction

Method used to map a dataset into a score, or a 1-dimensional projection of itself. The mapping should be modelled over the original (real) dataset.

alphafloat, default=0.05

Percentile used to determine the radius of the euclidean ball.

References

[1]

Alaa, A., Van Breugel, B., Saveliev, E. S., & van der Schaar, M. (2022, June). How faithful is your synthetic data? sample-level metrics for evaluating and auditing generative models. In International Conference on Machine Learning (pp. 290-306). PMLR.

Attributes:
center_float

Value of the center of the euclidean ball.


fit(X_real)[source]

Compute statistics necessary to calculate $alpha$-precision.

Parameters:
X_realarray-like or pd.DataFrame, shape (n_samples, n_features)

The real (original) dataset used to fit self.scorer_real.

Returns:
selfobject

Returns an instance of the class.

get_metadata_routing()

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:
routingMetadataRequest

A MetadataRequest encapsulating routing information.

score(X_synth)[source]

Returns 1 if a sample resides in the $alpha$-support of the original distribution, 0 otherwise.

Parameters:
Xarray-like or pd.DataFrame, shape (n_samples, n_features)

Input data over which $alpha$-precision will be calculated.

Returns:
scoresnp.ndarray, shape (n_samples,)

$alpha$-precision scores.

set_fit_request(*, X_real: bool | None | str = '$UNCHANGED$') → AlphaPrecision

Request metadata passed to the fit method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to fit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:
X_realstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for X_real parameter in fit.

Returns:
selfobject

The updated object.

set_score_request(*, X_synth: bool | None | str = '$UNCHANGED$') → AlphaPrecision

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to score.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:
X_synthstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for X_synth parameter in score.

Returns:
selfobject

The updated object.