GroupedTargetMetricVisualization#

class openstef_beam.analysis.visualizations.GroupedTargetMetricVisualization(**data: Any) → None[source]

Bases: VisualizationProvider

Creates bar charts and box plots for comparing metrics across targets and groups.

Generates interactive charts comparing performance metrics across different targets, model runs, or target groups. Supports deterministic metrics (MAE, RMSE) and quantile-based metrics (quantile losses) for performance comparisons.

Key features:

  • Bar charts for individual target comparisons

  • Box plots for grouped target analysis

  • Support for selector-based metrics (e.g., show metric at best-performing quantile)

  • Color-coded grouping for easy identification

  • Interactive tooltips with detailed metric values

Use cases:

  • Identify which targets are hardest to predict

  • Compare model performance across target categories

  • Analyze performance variations within target groups

  • Evaluate model consistency across different scenarios

Example

Comparing RMAE across targets for different models

>>> from openstef_beam.analysis import AnalysisConfig
>>> from openstef_beam.analysis.visualizations import GroupedTargetMetricVisualization
>>> from openstef_core.types import Quantile
>>>
>>> analysis_config = AnalysisConfig(
...     visualization_providers=[
...         # Compare median forecast accuracy
...         GroupedTargetMetricVisualization(
...             name="rmae_comparison",
...             metric="rMAE",
...             quantile=Quantile(0.5),
...         ),
...         # Compare overall probabilistic performance
...         GroupedTargetMetricVisualization(
...             name="rcrps_comparison",
...             metric="rCRPS",  # Global metric, no quantile needed
...         ),
...         # Show metric at best-performing quantile
...         GroupedTargetMetricVisualization(
...             name="adaptive_accuracy",
...             metric="rMAE",
...             selector_metric="rCRPS",  # Find quantile with best rCRPS
...         ),
...     ]
... )
Parameters:

data (Any)

metric: str
quantile: Quantile | None
selector_metric: str | None
property supported_aggregations: set[AnalysisAggregation]

Returns the set of aggregation types supported by this provider.

Returns:

Set of supported VisualizationAggregation values.

create_by_target(reports: list[tuple[TargetMetadata, EvaluationSubsetReport]]) → VisualizationOutput[source]

Creates visualization comparing multiple targets from the same run.

Groups reports by target metadata and creates visualizations showing performance differences across individual targets within the same model run.

Parameters:
Returns:

Visualization comparing performance across different targets.

Return type:

VisualizationOutput

create_by_run_and_none(reports: dict[RunName, list[tuple[TargetMetadata, EvaluationSubsetReport]]]) → VisualizationOutput[source]

Creates visualization comparing multiple runs on the same target group.

Groups reports by run_name and creates comparative visualizations showing how different models or configurations perform on the same targets.

Parameters:

  • reports (dict[TypeAliasType, list[tuple[TargetMetadata, EvaluationSubsetReport]]]) – Dictionary mapping run names to lists of (metadata, report) tuples for that run.

  • reports

Returns:

Visualization comparing different model runs on the same targets.

Return type:

VisualizationOutput

create_by_run_and_target(reports: dict[RunName, list[tuple[TargetMetadata, EvaluationSubsetReport]]]) → VisualizationOutput[source]

Creates visualization comparing multiple runs on the same target group.

Groups reports by run_name and creates comparative visualizations showing how different models or configurations perform on the same targets.

Parameters:

  • reports (dict[TypeAliasType, list[tuple[TargetMetadata, EvaluationSubsetReport]]]) – Dictionary mapping run names to lists of (metadata, report) tuples for that run.

  • reports

Returns:

Visualization comparing different model runs on the same targets.

Return type:

VisualizationOutput

create_by_run_and_group(reports: dict[tuple[RunName, GroupName], list[tuple[TargetMetadata, EvaluationSubsetReport]]]) → VisualizationOutput[source]

Creates visualization across multiple runs and target groups.

Creates matrix-style comparisons showing how different models perform across different target categories, enabling full comparative analysis.

Parameters:

  • reports (dict[tuple[TypeAliasType, TypeAliasType], list[tuple[TargetMetadata, EvaluationSubsetReport]]]) – Dictionary mapping (run_name, group_name) tuples to lists of (metadata, report) tuples for that combination.

  • reports

Returns:

Visualization matrix comparing runs across target groups.

Return type:

VisualizationOutput

create_by_group(reports: dict[GroupName, list[tuple[TargetMetadata, EvaluationSubsetReport]]]) → VisualizationOutput[source]

Creates visualization comparing multiple targets from the same run.

Groups targets by their group_name and creates comparative visualizations showing performance differences across target categories or types.

Parameters:

  • reports (dict[TypeAliasType, list[tuple[TargetMetadata, EvaluationSubsetReport]]]) – Dictionary mapping group names to lists of (metadata, report) tuples for that group.

  • reports

Returns:

Visualization comparing performance across different target groups.

Return type:

VisualizationOutput

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': False, 'extra': 'ignore', 'protected_namespaces': ()}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].