QuantileCalibrationBoxPlotter

class openstef_beam.analysis.plots.QuantileCalibrationBoxPlotter → None

Bases: object

Creates boxplots showing calibration error distributions across targets for each quantile.

This plotter visualizes how well probabilistic forecasts are calibrated by showing the distribution of calibration errors (observed - forecasted probability) across multiple targets for each quantile level. This helps identify:

• Which quantiles are systematically over/under-confident across targets

• The consistency of calibration across different targets

• Overall uncertainty estimation quality per quantile level

Well-calibrated forecasts will have boxplots centered around zero (no systematic bias) with tight distributions (consistent calibration across targets).

Example

Validating forecast calibration across multiple models and targets

>>> from openstef_core.types import Quantile
>>> plotter = QuantileCalibrationBoxPlotter()
>>>
>>> # Define common quantiles for all models/targets
>>> quantiles = [Quantile(0.1), Quantile(0.5), Quantile(0.9)]
>>>
>>> # ModelA: Well-calibrated model across different sites
>>> _ = plotter.add_model("ModelA", quantiles, [Quantile(0.08), Quantile(0.52), Quantile(0.88)])  # Site1
>>> _ = plotter.add_model("ModelA", quantiles, [Quantile(0.12), Quantile(0.48), Quantile(0.92)])  # Site2
>>> _ = plotter.add_model("ModelA", quantiles, [Quantile(0.09), Quantile(0.51), Quantile(0.89)])  # Site3
>>>
>>> # ModelB: Overconfident model across different sites
>>> _ = plotter.add_model("ModelB", quantiles, [Quantile(0.15), Quantile(0.55), Quantile(0.85)])  # Site1
>>> _ = plotter.add_model("ModelB", quantiles, [Quantile(0.18), Quantile(0.58), Quantile(0.82)])  # Site2
>>> _ = plotter.add_model("ModelB", quantiles, [Quantile(0.16), Quantile(0.57), Quantile(0.84)])  # Site3
>>>
>>> # Generate boxplot showing calibration error distributions
>>> fig = plotter.plot(title="Multi-Model Calibration Analysis")
>>> type(fig).__name__
'Figure'

__init__() → None

Initialize the plotter with empty model data storage.

add_model(model_name: str, forecasted_probs: list[Quantile], observed_probs: list[Quantile]) → QuantileCalibrationBoxPlotter

Add a model’s forecasted and observed probabilities to the plot.

Parameters:

• model_name (str) – The name of the model.

• forecasted_probs (list[Quantile]) – List of forecasted probabilities.

• observed_probs (list[Quantile]) – List of observed probabilities.

• model_name

• forecasted_probs

• observed_probs

Returns:

The current instance for method chaining.

Return type:

QuantileCalibrationBoxPlotter

Raises:

ValueError – If forecasted and observed probability lists have different lengths.

Return type:

QuantileCalibrationBoxPlotter

plot(title: str = 'Quantile Calibration Boxplot') → Figure

Create and return a quantile calibration boxplot for all added models.

Parameters:

• title (str) – Title of the plot.

• title

Returns:

The resulting plot.

Return type:

plotly.graph_objects.Figure

Raises:

ValueError – If no model data has been added.

Return type:

Figure