crps#

openstef_beam.metrics.crps(y_true: ndarray[tuple[Any, ...], dtype[floating]], y_pred: ndarray[tuple[Any, ...], dtype[floating]], quantiles: ndarray[tuple[Any, ...], dtype[floating]], sample_weights: ndarray[tuple[Any, ...], dtype[floating]] | None = None) → float[source]#

Calculate the Continuous Ranked Probability Score (CRPS) for probabilistic forecasts.

CRPS is a proper scoring rule that measures the quality of probabilistic forecasts. It generalizes the absolute error to distributional forecasts and is expressed in the same units as the forecast variable.

Parameters:

  • y_true (ndarray[tuple[Any, ...], dtype[floating]]) – Observed values with shape (num_samples,).

  • y_pred (ndarray[tuple[Any, ...], dtype[floating]]) – Predicted quantiles with shape (num_samples, num_quantiles). Each row contains quantile predictions for the corresponding observation.

  • quantiles (ndarray[tuple[Any, ...], dtype[floating]]) – Quantile levels with shape (num_quantiles,). Must be sorted in ascending order and contain values in [0, 1].

  • sample_weights (ndarray[tuple[Any, ...], dtype[floating]] | None) – Optional weights for each sample with shape (num_samples,). If None, all samples are weighted equally.

Returns:

The weighted average CRPS across all samples. Lower values indicate better forecast quality.

Return type:

float

Example

Evaluate quantile forecasts for energy load:

>>> import numpy as np
>>> y_true = np.array([100, 120, 110])
>>> quantiles = np.array([0.1, 0.5, 0.9])
>>> y_pred = np.array([[95, 100, 105],    # Quantiles for first observation
...                    [115, 120, 125],   # Quantiles for second observation
...                    [105, 110, 115]])  # Quantiles for third observation
>>> score = crps(y_true, y_pred, quantiles)
>>> isinstance(score, float)
True

Note

CRPS reduces to the absolute error when comparing point forecasts (single quantile). For well-calibrated forecasts, CRPS approximately equals half the expected absolute error of random forecasts.

Raises:

MissingExtraError – If the scoringrules package is not installed.

Parameters:

  • y_true (ndarray[tuple[Any, ...], dtype[floating]])

  • y_pred (ndarray[tuple[Any, ...], dtype[floating]])

  • quantiles (ndarray[tuple[Any, ...], dtype[floating]])

  • sample_weights (ndarray[tuple[Any, ...], dtype[floating]] | None)

Return type:

float