RCSProvider#

class openstef_beam.evaluation.metric_providers.RCSProvider(**data: Any) → None[source]

Bases: MetricProvider

Provides Regression Coverage Score metrics.

Measures the fraction of observed values inside symmetric prediction intervals, such as P10-P90. For each quantile, finds its symmetric counterpart and computes RCS between them.

Parameters:

data (Any)

property metric_names: frozenset[str]

Declared metric names that this provider produces.

Override in subclasses to enable eager metric-name validation (e.g. in the hyperparameter tuner).

compute_probabilistic(y_true: NDArray[floating], y_pred: NDArray[floating], quantiles: list[Quantile]) → dict[Quantile | Literal['global'], dict[str, Annotated[float, BeforeValidator(func=_convert_none_to_nan, json_schema_input_type=PydanticUndefined)]]][source]

Compute RCS for each quantile by finding its symmetric counterpart.

For each quantile q, finds the symmetric quantile (1-q) and computes RCS between them. Only processes quantiles for which a symmetric counterpart is available.

Parameters:

  • y_true (NDArray[floating]) – True values, 1D array of shape (num_samples,).

  • y_pred (NDArray[floating]) – Predicted values, 2D array of shape (num_samples, num_quantiles).

  • quantiles (list[Quantile]) – Quantiles used for prediction, sequence of length (num_quantiles,).

  • y_true

  • y_pred

  • quantiles

Returns:

QuantileMetricsDict containing RCS metrics for each processable quantile.

Return type:

dict[Union[Quantile, Literal['global']], dict[str, float]]

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': False, 'extra': 'ignore', 'protected_namespaces': ()}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].