mlresearch.metrics.AlphaPrecision

class mlresearch.metrics.AlphaPrecision(scorer_real, alpha=0.05)[source]

Measures synthetic data fidelity. It estimates the probability that a synthetic sample resides in the $alpha$-support of the real distribution.

This is an implementation of the metric proposed in [1].

Warning

This metric is not listed in the get_scorer_names function since it is following an unconventional structure.

Parameters:
scorer_realfunction

Method used to map a dataset into a score, or a 1-dimensional projection of itself. The mapping should be modelled over the original (real) dataset.

alphafloat, default=0.05

Percentile used to determine the radius of the euclidean ball.

Attributes:
center_float

Value of the center of the euclidean ball.

References

[1]

Alaa, A., Van Breugel, B., Saveliev, E. S., & van der Schaar, M. (2022, June). How faithful is your synthetic data? sample-level metrics for evaluating and auditing generative models. In International Conference on Machine Learning (pp. 290-306). PMLR.


fit(X_real)[source]

Compute statistics necessary to calculate $alpha$-precision.

Parameters:
X_realarray-like or pd.DataFrame, shape (n_samples, n_features)

The real (original) dataset used to fit self.scorer_real.

Returns:
selfobject

Returns an instance of the class.

get_metadata_routing()

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:
routingMetadataRequest

A MetadataRequest encapsulating routing information.

score(X_synth)[source]

Returns 1 if a sample resides in the $alpha$-support of the original distribution, 0 otherwise.

Parameters:
Xarray-like or pd.DataFrame, shape (n_samples, n_features)

Input data over which $alpha$-precision will be calculated.

Returns:
scoresnp.ndarray, shape (n_samples,)

$alpha$-precision scores.

set_fit_request(*, X_real: bool | None | str = '$UNCHANGED$') → AlphaPrecision

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to fit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:
X_realstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for X_real parameter in fit.

Returns:
selfobject

The updated object.

set_score_request(*, X_synth: bool | None | str = '$UNCHANGED$') → AlphaPrecision

Configure whether metadata should be requested to be passed to the score method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to score.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:
X_synthstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for X_synth parameter in score.

Returns:
selfobject

The updated object.