QuantileProbabilityVisualization#

class openstef_beam.analysis.visualizations.QuantileProbabilityVisualization(**data: Any) → None[source]

Bases: VisualizationProvider

Creates calibration plots comparing observed vs forecasted probabilities.

Evaluates calibration quality of probabilistic forecasts by plotting observed frequencies against forecasted probabilities for different quantile levels. Perfect calibration shows points along the diagonal where observed probability equals forecasted probability.

Identifies forecast issues:

  • Overconfident predictions (points below diagonal)

  • Underconfident predictions (points above diagonal)

  • Systematic biases in uncertainty estimation

  • Overall forecast reliability across quantile ranges

Supports comparison across different model runs, targets, and aggregation levels to evaluate which models provide better calibrated uncertainty estimates.

Example

Basic usage in analysis pipeline

>>> from openstef_beam.analysis import AnalysisConfig
>>> from openstef_beam.analysis.visualizations import QuantileProbabilityVisualization
>>>
>>> # Configure probability calibration analysis
>>> analysis_config = AnalysisConfig(
...     visualization_providers=[
...         QuantileProbabilityVisualization(
...             name="probability_calibration",
...         ),
...     ]
... )
>>>
>>> # The visualization will generate calibration plots showing
>>> # observed vs forecasted probabilities for model evaluation
Parameters:

data (Any)

property supported_aggregations: set[AnalysisAggregation]

Returns the set of aggregation types supported by this provider.

Returns:

Set of supported VisualizationAggregation values.

create_by_none(report: EvaluationSubsetReport, metadata: TargetMetadata) → VisualizationOutput[source]

Creates visualization for a single target from a single run.

Generates detailed analysis for individual target performance, typically showing time series, detailed metrics, or target-specific insights.

Returns:

Visualization focused on the specific target’s performance.

Parameters:
Return type:

VisualizationOutput

create_by_run_and_none(reports: dict[RunName, list[tuple[TargetMetadata, EvaluationSubsetReport]]]) → VisualizationOutput[source]

Creates visualization comparing multiple runs on the same target group.

Groups reports by run_name and creates comparative visualizations showing how different models or configurations perform on the same targets.

Parameters:

  • reports (dict[TypeAliasType, list[tuple[TargetMetadata, EvaluationSubsetReport]]]) – Dictionary mapping run names to lists of (metadata, report) tuples for that run.

  • reports

Returns:

Visualization comparing different model runs on the same targets.

Return type:

VisualizationOutput

create_by_run_and_target(reports: dict[RunName, list[tuple[TargetMetadata, EvaluationSubsetReport]]]) → VisualizationOutput[source]

Creates visualization comparing multiple runs on the same target group.

Groups reports by run_name and creates comparative visualizations showing how different models or configurations perform on the same targets.

Parameters:

  • reports (dict[TypeAliasType, list[tuple[TargetMetadata, EvaluationSubsetReport]]]) – Dictionary mapping run names to lists of (metadata, report) tuples for that run.

  • reports

Returns:

Visualization comparing different model runs on the same targets.

Return type:

VisualizationOutput

create_by_target(reports: list[tuple[TargetMetadata, EvaluationSubsetReport]]) → VisualizationOutput[source]

Creates visualization comparing multiple targets from the same run.

Groups reports by target metadata and creates visualizations showing performance differences across individual targets within the same model run.

Parameters:
Returns:

Visualization comparing performance across different targets.

Return type:

VisualizationOutput

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': False, 'extra': 'ignore', 'protected_namespaces': ()}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].