LocalBenchmarkStorage#

class openstef_beam.benchmarking.LocalBenchmarkStorage(base_path: Path, *, skip_when_existing: bool = True, predictions_filename: str = 'predictions.parquet', backtest_dirname: str = 'backtest', evaluations_dirname: str = 'evaluation', analysis_dirname: str = 'analysis')[source]#

Bases: BenchmarkStorage

File system-based storage implementation for benchmark results.

Stores benchmark artifacts (predictions, evaluations, and visualizations) in a structured directory hierarchy on the local file system. Supports conditional skipping of existing files to avoid redundant processing.

Directory structure:

base_path/ ├── backtest/ │ └── group_name/ │ └── target_name/ │ └── predictions.parquet ├── evaluation/ │ └── group_name/ │ └── target_name/ └── analysis/

├── group_name/ │ ├── target_name/ # Target-specific visualizations │ └── global/ # Group-level aggregated visualizations └── global/ # Global aggregated visualizations

Parameters:

  • base_path (Path)

  • skip_when_existing (bool)

  • predictions_filename (str)

  • backtest_dirname (str)

  • evaluations_dirname (str)

  • analysis_dirname (str)

__init__(base_path: Path, *, skip_when_existing: bool = True, predictions_filename: str = 'predictions.parquet', backtest_dirname: str = 'backtest', evaluations_dirname: str = 'evaluation', analysis_dirname: str = 'analysis')[source]#

Initialize local file system storage.

Parameters:

  • base_path (Path) – Root directory where all benchmark artifacts will be stored.

  • skip_when_existing (bool) – When True, has_* methods consider existing files as valid and skip reprocessing. When False, always indicates missing data.

  • predictions_filename (str) – Name of the parquet file for storing backtest predictions.

  • backtest_dirname (str) – Directory name for backtest predictions within base_path.

  • evaluations_dirname (str) – Directory name for evaluation reports within each target.

  • analysis_dirname (str) – Directory name for analysis visualizations.

  • base_path

  • skip_when_existing

  • predictions_filename

  • backtest_dirname

  • evaluations_dirname

  • analysis_dirname

save_backtest_output(target: BenchmarkTarget, output: TimeSeriesDataset) → None[source]#

Save backtest predictions to a parquet file.

Parameters:
Return type:

None

load_backtest_output(target: BenchmarkTarget) → TimeSeriesDataset[source]#

Load backtest predictions from a parquet file.

Returns:

The loaded prediction data.

Return type:

TimeSeriesDataset

Parameters:

target (BenchmarkTarget)

Return type:

TimeSeriesDataset

has_backtest_output(target: BenchmarkTarget) → bool[source]#

Check if backtest output exists for the given benchmark target.

Returns:

True if backtest output is stored for the target, False otherwise.

Parameters:

target (BenchmarkTarget)

Return type:

bool

save_evaluation_output(target: BenchmarkTarget, output: EvaluationReport) → None[source]#

Save the evaluation report for a specific benchmark target.

Stores the evaluation metrics and analysis results, associating them with the target configuration. Must handle overwrites of existing data gracefully.

Parameters:
Return type:

None

load_evaluation_output(target: BenchmarkTarget) → EvaluationReport[source]#

Load previously saved evaluation report for a benchmark target.

Returns:

The stored evaluation report containing metrics and analysis results.

Raises:

KeyError – When no evaluation output exists for the given target.

Parameters:

target (BenchmarkTarget)

Return type:

EvaluationReport

has_evaluation_output(target: BenchmarkTarget) → bool[source]#

Check if evaluation output exists for the given benchmark target.

Returns:

True if evaluation output is stored for the target, False otherwise.

Parameters:

target (BenchmarkTarget)

Return type:

bool

save_analysis_output(output: AnalysisOutput) → None[source]#

Save analysis visualizations to HTML files.

Parameters:

output (AnalysisOutput)

Return type:

None

has_analysis_output(scope: AnalysisScope) → bool[source]#

Check if analysis output exists for the given target or global scope.

Parameters:

  • scope (AnalysisScope) – The scope of the analysis output to check.

  • scope

Returns:

True if analysis output exists for the specified scope, False otherwise.

Return type:

bool

get_predictions_path_for_target(target: BenchmarkTarget) → Path[source]#

Returns the path for storing predictions for a target.

Parameters:

target (BenchmarkTarget)

Return type:

Path

get_evaluations_path_for_target(target: BenchmarkTarget) → Path[source]#

Returns the path for storing evaluation results for a target.

Parameters:

target (BenchmarkTarget)

Return type:

Path

get_analysis_path(scope: AnalysisScope) → Path[source]#

Get the file path for storing analysis output based on aggregation scope.

Returns:

Directory path where analysis results should be stored.

Return type:

Path

Parameters:

scope (AnalysisScope)

Return type:

Path