WindowedMetricVisualization#

class openstef_beam.analysis.visualizations.WindowedMetricVisualization(**data: Any) → None[source]

Bases: VisualizationProvider

Creates time series plots showing metric evolution across evaluation windows.

Displays how evaluation metrics change over time by plotting metric values on a timeline where each point represents performance over a specific time window. The visualization reveals performance trends, seasonal patterns, and helps identify periods where model accuracy degrades or improves.

What you’ll see:

  • Time series line plot with metric values on Y-axis and time on X-axis

  • Each point shows metric computed over a sliding evaluation window

  • Multiple lines when comparing across targets or model runs

  • Clear trends showing model performance stability over time

Useful for identifying:

  • Performance degradation patterns over time

  • Seasonal effects in forecasting accuracy

  • Model stability across different periods

  • Optimal retraining intervals based on performance drops

Example

>>> from openstef_beam.analysis import AnalysisConfig
>>> from openstef_beam.analysis.visualizations import WindowedMetricVisualization
>>> from openstef_beam.evaluation import Window
>>> from datetime import timedelta
>>>
>>> analysis_config = AnalysisConfig(
...     visualization_providers=[
...         WindowedMetricVisualization(
...             name="mae_evolution",
...             metric="MAE",
...             window=Window(lag=timedelta(hours=0), size=timedelta(days=7)),
...         ),
...     ]
... )
Parameters:

data (Any)

metric: TypeAliasType
window: Window
property supported_aggregations: set[AnalysisAggregation]

Returns the set of aggregation types supported by this provider.

Returns:

Set of supported VisualizationAggregation values.

create_by_none(report: EvaluationSubsetReport, metadata: TargetMetadata) → VisualizationOutput[source]

Creates visualization for a single target from a single run.

Generates detailed analysis for individual target performance, typically showing time series, detailed metrics, or target-specific insights.

Returns:

Visualization focused on the specific target’s performance.

Parameters:
Return type:

VisualizationOutput

create_by_run_and_none(reports: dict[RunName, list[tuple[TargetMetadata, EvaluationSubsetReport]]]) → VisualizationOutput[source]

Creates visualization comparing multiple runs on the same target group.

Groups reports by run_name and creates comparative visualizations showing how different models or configurations perform on the same targets.

Parameters:

  • reports (dict[TypeAliasType, list[tuple[TargetMetadata, EvaluationSubsetReport]]]) – Dictionary mapping run names to lists of (metadata, report) tuples for that run.

  • reports

Returns:

Visualization comparing different model runs on the same targets.

Return type:

VisualizationOutput

create_by_target(reports: list[tuple[TargetMetadata, EvaluationSubsetReport]]) → VisualizationOutput[source]

Creates visualization comparing multiple targets from the same run.

Groups reports by target metadata and creates visualizations showing performance differences across individual targets within the same model run.

Parameters:
Returns:

Visualization comparing performance across different targets.

Return type:

VisualizationOutput

create_by_run_and_target(reports: dict[RunName, list[tuple[TargetMetadata, EvaluationSubsetReport]]]) → VisualizationOutput[source]

Creates visualization comparing multiple runs on the same target group.

Groups reports by run_name and creates comparative visualizations showing how different models or configurations perform on the same targets.

Parameters:

  • reports (dict[TypeAliasType, list[tuple[TargetMetadata, EvaluationSubsetReport]]]) – Dictionary mapping run names to lists of (metadata, report) tuples for that run.

  • reports

Returns:

Visualization comparing different model runs on the same targets.

Return type:

VisualizationOutput

create_by_run_and_group(reports: dict[tuple[RunName, GroupName], list[tuple[TargetMetadata, EvaluationSubsetReport]]]) → VisualizationOutput[source]

Creates visualization across multiple runs and target groups.

Creates matrix-style comparisons showing how different models perform across different target categories, enabling full comparative analysis.

Parameters:

  • reports (dict[tuple[TypeAliasType, TypeAliasType], list[tuple[TargetMetadata, EvaluationSubsetReport]]]) – Dictionary mapping (run_name, group_name) tuples to lists of (metadata, report) tuples for that combination.

  • reports

Returns:

Visualization matrix comparing runs across target groups.

Return type:

VisualizationOutput

create_by_group(reports: dict[GroupName, list[tuple[TargetMetadata, EvaluationSubsetReport]]]) → VisualizationOutput[source]

Creates visualization comparing multiple targets from the same run.

Groups targets by their group_name and creates comparative visualizations showing performance differences across target categories or types.

Parameters:

  • reports (dict[TypeAliasType, list[tuple[TargetMetadata, EvaluationSubsetReport]]]) – Dictionary mapping group names to lists of (metadata, report) tuples for that group.

  • reports

Returns:

Visualization comparing performance across different target groups.

Return type:

VisualizationOutput

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': False, 'extra': 'ignore', 'protected_namespaces': ()}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].