QuantileProbabilityVisualization#

class openstef_beam.analysis.visualizations.QuantileProbabilityVisualization(*, name: str)[source]

Bases: VisualizationProvider

Creates calibration plots comparing observed vs forecasted probabilities.

Evaluates calibration quality of probabilistic forecasts by plotting observed frequencies against forecasted probabilities for different quantile levels. Perfect calibration shows points along the diagonal where observed probability equals forecasted probability.

Identifies forecast issues:

  • Overconfident predictions (points below diagonal)

  • Underconfident predictions (points above diagonal)

  • Systematic biases in uncertainty estimation

  • Overall forecast reliability across quantile ranges

Supports comparison across different model runs, targets, and aggregation levels to evaluate which models provide better calibrated uncertainty estimates.

Example

Basic usage in analysis pipeline

>>> from openstef_beam.analysis import AnalysisConfig
>>> from openstef_beam.analysis.visualizations import QuantileProbabilityVisualization
>>>
>>> # Configure probability calibration analysis
>>> analysis_config = AnalysisConfig(
...     visualization_providers=[
...         QuantileProbabilityVisualization(
...             name="probability_calibration",
...         ),
...     ]
... )
>>>
>>> # The visualization will generate calibration plots showing
>>> # observed vs forecasted probabilities for model evaluation
Parameters:

name (str)

property supported_aggregations: set[AnalysisAggregation]

Returns the set of aggregation types supported by this provider.

Returns:

Set of supported VisualizationAggregation values.

Overrides:

VisualizationProvider.supported_aggregations

create_by_none(report: EvaluationSubsetReport, metadata: TargetMetadata) → VisualizationOutput[source]

Creates visualization for a single target from a single run.

Generates detailed analysis for individual target performance, typically showing time series, detailed metrics, or target-specific insights.

Return type:

VisualizationOutput

Returns:

Visualization focused on the specific target’s performance.

Parameters:
Overrides:

VisualizationProvider.create_by_none

create_by_run_and_none(reports: dict[RunName, list[tuple[TargetMetadata, EvaluationSubsetReport]]]) → VisualizationOutput[source]

Creates visualization comparing multiple runs on the same target group.

Groups reports by run_name and creates comparative visualizations showing how different models or configurations perform on the same targets.

Parameters:

reports (dict[str, list[tuple[TargetMetadata, EvaluationSubsetReport]]]) – Dictionary mapping run names to lists of (metadata, report) tuples for that run.

Return type:

VisualizationOutput

Returns:

Visualization comparing different model runs on the same targets.

Overrides:

VisualizationProvider.create_by_run_and_none

create_by_run_and_target(reports: dict[RunName, list[tuple[TargetMetadata, EvaluationSubsetReport]]]) → VisualizationOutput[source]

Creates visualization comparing multiple runs on the same target group.

Groups reports by run_name and creates comparative visualizations showing how different models or configurations perform on the same targets.

Parameters:

reports (dict[str, list[tuple[TargetMetadata, EvaluationSubsetReport]]]) – Dictionary mapping run names to lists of (metadata, report) tuples for that run.

Return type:

VisualizationOutput

Returns:

Visualization comparing different model runs on the same targets.

Overrides:

VisualizationProvider.create_by_run_and_target

create_by_target(reports: list[tuple[TargetMetadata, EvaluationSubsetReport]]) → VisualizationOutput[source]

Creates visualization comparing multiple targets from the same run.

Groups reports by target metadata and creates visualizations showing performance differences across individual targets within the same model run.

Parameters:

reports (list[tuple[TargetMetadata, EvaluationSubsetReport]]) – List of (metadata, report) tuples for each target in the run.

Return type:

VisualizationOutput

Returns:

Visualization comparing performance across different targets.

Overrides:

VisualizationProvider.create_by_target

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': False, 'extra': 'ignore', 'protected_namespaces': ()}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].