openstef package

Subpackages

• openstef.data_classes package
  • Submodules
  • openstef.data_classes.data_prep module
    • DataPrepDataClass
      • DataPrepDataClass.arguments
      • DataPrepDataClass.klass
      • DataPrepDataClass.load()
  • openstef.data_classes.model_specifications module
    • ModelSpecificationDataClass
      • ModelSpecificationDataClass.feature_modules
      • ModelSpecificationDataClass.feature_names
      • ModelSpecificationDataClass.hyper_params
      • ModelSpecificationDataClass.id
  • openstef.data_classes.prediction_job module
    • PredictionJobDataClass
      • PredictionJobDataClass.Config
      • PredictionJobDataClass.alternative_forecast_model_pid
      • PredictionJobDataClass.backtest_split_func
      • PredictionJobDataClass.completeness_threshold
      • PredictionJobDataClass.data_prep_class
      • PredictionJobDataClass.default_modelspecs
      • PredictionJobDataClass.depends_on
      • PredictionJobDataClass.description
      • PredictionJobDataClass.flatliner_threshold_minutes
      • PredictionJobDataClass.forecast_type
      • PredictionJobDataClass.get()
      • PredictionJobDataClass.horizon_minutes
      • PredictionJobDataClass.hub_height
      • PredictionJobDataClass.id
      • PredictionJobDataClass.lat
      • PredictionJobDataClass.lon
      • PredictionJobDataClass.minimal_table_length
      • PredictionJobDataClass.model
      • PredictionJobDataClass.n_turbines
      • PredictionJobDataClass.name
      • PredictionJobDataClass.pipelines_to_run
      • PredictionJobDataClass.quantiles
      • PredictionJobDataClass.resolution_minutes
      • PredictionJobDataClass.save_train_forecasts
      • PredictionJobDataClass.sid
      • PredictionJobDataClass.train_components
      • PredictionJobDataClass.train_horizons_minutes
      • PredictionJobDataClass.train_split_func
      • PredictionJobDataClass.turbine_type
  • openstef.data_classes.split_function module
    • SplitFuncDataClass
      • SplitFuncDataClass.arguments
      • SplitFuncDataClass.function
      • SplitFuncDataClass.load()
  • Module contents
• openstef.feature_engineering package
  • Submodules
  • openstef.feature_engineering.apply_features module
    • apply_features()
  • openstef.feature_engineering.data_preparation module
    • ARDataPreparation
      • ARDataPreparation.prepare_forecast_data()
      • ARDataPreparation.prepare_train_data()
    • AbstractDataPreparation
      • AbstractDataPreparation.check_model()
      • AbstractDataPreparation.prepare_forecast_data()
      • AbstractDataPreparation.prepare_train_data()
    • LegacyDataPreparation
      • LegacyDataPreparation.prepare_forecast_data()
      • LegacyDataPreparation.prepare_train_data()
  • openstef.feature_engineering.feature_adder module
    • FeatureAdder
      • FeatureAdder.apply_features()
      • FeatureAdder.name
      • FeatureAdder.parse_feature_name()
      • FeatureAdder.required_features()
    • FeatureDispatcher
      • FeatureDispatcher.apply_features()
      • FeatureDispatcher.dispatch_features()
    • ParsedFeature
      • ParsedFeature.name
      • ParsedFeature.params
    • adders_from_module()
    • adders_from_modules()
  • openstef.feature_engineering.feature_applicator module
    • AbstractFeatureApplicator
      • AbstractFeatureApplicator.add_features()
    • OperationalPredictFeatureApplicator
      • OperationalPredictFeatureApplicator.add_features()
    • TrainFeatureApplicator
      • TrainFeatureApplicator.add_features()
  • openstef.feature_engineering.general module
    • add_missing_feature_columns()
    • enforce_feature_order()
    • remove_non_requested_feature_columns()
  • openstef.feature_engineering.historic_features module
    • add_historic_load_as_a_feature()
  • openstef.feature_engineering.holiday_features module
    • check_for_bridge_day()
    • generate_holiday_feature_functions()
  • openstef.feature_engineering.lag_features module
    • extract_lag_features()
    • generate_lag_feature_functions()
    • generate_non_trivial_lag_times()
    • generate_trivial_lag_features()
  • openstef.feature_engineering.weather_features module
    • add_additional_solar_features()
    • add_additional_wind_features()
    • add_humidity_features()
    • calc_air_density()
    • calc_dewpoint()
    • calc_saturation_pressure()
    • calc_vapour_pressure()
    • calculate_dni()
    • calculate_gti()
    • calculate_windspeed_at_hubheight()
    • calculate_windturbine_power_output()
    • humidity_calculations()
  • Module contents
• openstef.metrics package
  • Submodules
  • openstef.metrics.figure module
    • convert_to_base64_data_uri()
    • plot_data_series()
    • plot_feature_importance()
  • openstef.metrics.metrics module
    • bias()
    • frac_in_stdev()
    • franks_skill_score()
    • franks_skill_score_peaks()
    • get_eval_metric_function()
    • mae()
    • nsme()
    • r_mae()
    • r_mae_highest()
    • r_mae_lowest()
    • r_mne_highest()
    • r_mpe_highest()
    • rmse()
    • skill_score()
    • skill_score_positive_peaks()
    • xgb_quantile_eval()
    • xgb_quantile_obj()
  • openstef.metrics.reporter module
    • Report
    • Reporter
      • Reporter.generate_report()
      • Reporter.get_fiabilities()
      • Reporter.get_metrics()
      • Reporter.write_report_to_disk()
  • Module contents
• openstef.model package
  • Subpackages
    • openstef.model.metamodels package
      • Submodules
      • openstef.model.metamodels.grouped_regressor module
      • openstef.model.metamodels.missing_values_handler module
      • Module contents
    • openstef.model.regressors package
      • Submodules
      • openstef.model.regressors.arima module
      • openstef.model.regressors.custom_regressor module
      • openstef.model.regressors.dazls module
      • openstef.model.regressors.lgbm module
      • openstef.model.regressors.linear module
      • openstef.model.regressors.proloaf module
      • openstef.model.regressors.regressor module
      • openstef.model.regressors.xgb module
      • openstef.model.regressors.xgb_quantile module
      • Module contents
  • Submodules
  • openstef.model.basecase module
    • BaseCaseModel
      • BaseCaseModel.can_predict_quantiles
      • BaseCaseModel.fit()
      • BaseCaseModel.make_basecase_forecast()
      • BaseCaseModel.predict()
      • BaseCaseModel.set_predict_request()
      • BaseCaseModel.set_score_request()
  • openstef.model.confidence_interval_applicator module
    • ConfidenceIntervalApplicator
      • ConfidenceIntervalApplicator.add_confidence_interval()
  • openstef.model.fallback module
    • generate_fallback()
  • openstef.model.model_creator module
    • ModelCreator
      • ModelCreator.MODEL_CONSTRUCTORS
      • ModelCreator.create_model()
  • openstef.model.objective module
    • ARIMARegressorObjective
      • ARIMARegressorObjective.get_params()
    • LGBRegressorObjective
      • LGBRegressorObjective.get_params()
      • LGBRegressorObjective.get_pruning_callback()
    • LinearRegressorObjective
      • LinearRegressorObjective.get_params()
    • ProLoafRegressorObjective
      • ProLoafRegressorObjective.get_params()
      • ProLoafRegressorObjective.get_pruning_callback()
    • RegressorObjective
      • RegressorObjective.create_report()
      • RegressorObjective.get_default_values()
      • RegressorObjective.get_params()
      • RegressorObjective.get_pruning_callback()
      • RegressorObjective.get_trial_track()
    • XGBQuantileRegressorObjective
      • XGBQuantileRegressorObjective.get_params()
      • XGBQuantileRegressorObjective.get_pruning_callback()
    • XGBRegressorObjective
      • XGBRegressorObjective.get_default_values()
      • XGBRegressorObjective.get_params()
      • XGBRegressorObjective.get_pruning_callback()
  • openstef.model.objective_creator module
    • ObjectiveCreator
      • ObjectiveCreator.OBJECTIVES
      • ObjectiveCreator.create_objective()
  • openstef.model.serializer module
    • MLflowSerializer
      • MLflowSerializer.get_model_age()
      • MLflowSerializer.load_model()
      • MLflowSerializer.remove_old_models()
      • MLflowSerializer.save_model()
  • openstef.model.standard_deviation_generator module
    • StandardDeviationGenerator
      • StandardDeviationGenerator.generate_standard_deviation_data()
  • Module contents
• openstef.model_selection package
  • Submodules
  • openstef.model_selection.model_selection module
    • backtest_split_default()
    • group_kfold()
    • random_sample()
    • sample_indices_train_val()
    • split_data_train_validation_test()
  • Module contents
• openstef.monitoring package
  • Submodules
  • openstef.monitoring.performance_meter module
    • PerformanceMeter
      • PerformanceMeter.checkpoint()
      • PerformanceMeter.complete_level()
      • PerformanceMeter.start_level()
  • openstef.monitoring.teams module
    • build_sql_query_string()
    • format_message()
    • get_card_section()
    • post_teams()
  • Module contents
• openstef.pipeline package
  • Submodules
  • openstef.pipeline.create_basecase_forecast module
    • create_basecase_forecast_pipeline()
    • generate_basecase_confidence_interval()
  • openstef.pipeline.create_component_forecast module
    • create_components_forecast_pipeline()
    • create_input()
  • openstef.pipeline.create_forecast module
    • create_forecast_pipeline()
    • create_forecast_pipeline_core()
  • openstef.pipeline.optimize_hyperparameters module
    • optimize_hyperparameters_pipeline()
    • optimize_hyperparameters_pipeline_core()
    • optuna_optimization()
  • openstef.pipeline.train_create_forecast_backtest module
    • train_model_and_forecast_back_test()
    • train_model_and_forecast_test_core()
  • openstef.pipeline.train_model module
    • train_model_pipeline()
    • train_model_pipeline_core()
    • train_pipeline_common()
    • train_pipeline_step_compute_features()
    • train_pipeline_step_load_model()
    • train_pipeline_step_split_data()
    • train_pipeline_step_train_model()
  • openstef.pipeline.utils module
    • generate_forecast_datetime_range()
  • Module contents
• openstef.postprocessing package
  • Submodules
  • openstef.postprocessing.postprocessing module
    • add_components_base_case_forecast()
    • add_prediction_job_properties_to_forecast()
    • calculate_wind_power()
    • normalize_and_convert_weather_data_for_splitting()
    • post_process_wind_solar()
    • split_forecast_in_components()
  • Module contents
• openstef.preprocessing package
  • Submodules
  • openstef.preprocessing.preprocessing module
    • replace_repeated_values_with_nan()
  • Module contents
• openstef.tasks package
  • Subpackages
    • openstef.tasks.utils package
      • Submodules
      • openstef.tasks.utils.dependencies module
      • openstef.tasks.utils.predictionjobloop module
      • openstef.tasks.utils.taskcontext module
      • Module contents
  • Submodules
  • openstef.tasks.calculate_kpi module
    • calc_kpi_for_specific_pid()
    • check_kpi_task()
    • main()
    • set_incomplete_kpi_to_nan()
  • openstef.tasks.create_basecase_forecast module
    • create_basecase_forecast_task()
    • main()
  • openstef.tasks.create_components_forecast module
    • create_components_forecast_task()
    • main()
  • openstef.tasks.create_forecast module
    • create_forecast_task()
    • main()
  • openstef.tasks.create_solar_forecast module
    • apply_fit_insol()
    • apply_persistence()
    • calc_norm()
    • combine_forecasts()
    • fides()
    • main()
    • make_solar_prediction_pj()
  • openstef.tasks.create_wind_forecast module
    • main()
    • make_wind_forecast_pj()
  • openstef.tasks.optimize_hyperparameters module
    • main()
    • optimize_hyperparameters_task()
  • openstef.tasks.run_tracy module
    • main()
    • run_tracy()
    • run_tracy_job()
  • openstef.tasks.split_forecast module
    • convert_coefdict_to_coefsdf()
    • determine_invalid_coefs()
    • find_components()
    • main()
    • split_forecast_task()
  • openstef.tasks.train_model module
    • main()
    • train_model_task()
  • Module contents
• openstef.validation package
  • Submodules
  • openstef.validation.validation module
    • calc_completeness_dataframe()
    • calc_completeness_features()
    • detect_ongoing_zero_flatliner()
    • drop_target_na()
    • is_data_sufficient()
    • validate()
  • Module contents

Submodules

openstef.enums module

class openstef.enums.ForecastType(value)

Bases: Enum

An enumeration.

BASECASE = 'basecase'

DEMAND = 'demand'

SOLAR = 'solar'

WIND = 'wind'

class openstef.enums.MLModelType(value)

Bases: Enum

An enumeration.

ARIMA = 'arima'

LGB = 'lgb'

LINEAR = 'linear'

ProLoaf = 'proloaf'

XGB = 'xgb'

XGB_QUANTILE = 'xgb_quantile'

class openstef.enums.PipelineType(value)

Bases: Enum

An enumeration.

FORECAST = 'forecast'

HYPER_PARMATERS = 'hyper_parameters'

TRAIN = 'train'

class openstef.enums.TracyJobResult(value)

Bases: Enum

An enumeration.

FAILED = 'failed'

SUCCESS = 'success'

UNKNOWN = 'unknown'

openstef.exceptions module

Openstef custom exceptions.

exception openstef.exceptions.ComponentForecastTooShortHorizonError

Bases: Exception

Component forecasts should be available for at least 30 hours in advance.

exception openstef.exceptions.InputDataInsufficientError

Bases: InputDataInvalidError

Insufficient input data.

exception openstef.exceptions.InputDataInvalidError

Bases: Exception

Invalid input data.

exception openstef.exceptions.InputDataOngoingZeroFlatlinerError

Bases: InputDataInvalidError

All recent load measurements are zero.

exception openstef.exceptions.InputDataWrongColumnOrderError

Bases: InputDataInvalidError

Wrong column order input data.

exception openstef.exceptions.ModelWithoutStDev

Bases: Exception

A machine learning model should have a valid standard deviation.

exception openstef.exceptions.NoPredictedLoadError(pid, message='No predicted load found')

Bases: Exception

No predicted load for given datatime range.

exception openstef.exceptions.NoRealisedLoadError(pid, message='No realised load found')

Bases: Exception

No realised load for given datetime range.

exception openstef.exceptions.OldModelHigherScoreError

Bases: Exception

Old model has a higher score then new model.

exception openstef.exceptions.PredictionJobException(metrics=None)

Bases: Exception

One or more prediction jobs raised an exception.

exception openstef.exceptions.SkipSaveTrainingForecasts

Bases: Exception

If old model is better or too young, you don’t need to save the traing forcast.

Module contents