openstef.model package#

Subpackages#

Submodules#

openstef.model.basecase module#

class openstef.model.basecase.BaseCaseModel#

Bases: BaseEstimator, RegressorMixin

property can_predict_quantiles#
fit()#
static make_basecase_forecast(forecast_input_data, overwrite_delay_hours=48)#

Make a basecase forecast.

The idea of the basecase forecast is that if all else fails, this forecasts is still available. Basecase example: the load of last week.

Parameters:

  • forecast_input_data (DataFrame) – Forecast input dataframe

  • overwrite_delay_hours (int) – times before this in the future are not forecasted

Raises:

  • ValueError – if columns T-7d or T-14d is not present

  • ValueError – If the start of the forecast is before the horizon of the regular forecast

Return type:

DataFrame

Returns:

Basecase forecast

predict(forecast_input_data)#

Predict using the basecase method. The basecase forecast is determined by the T-7d and T-14d load.

This means fitting the model is not required. However a fit method is still included to be fully comatible with sklearn.

Parameters:

forecast_input_data (DataFrame) – Forecast input dataframe

Return type:

DataFrame

Returns:

Basecase forecast

openstef.model.confidence_interval_applicator module#

class openstef.model.confidence_interval_applicator.ConfidenceIntervalApplicator(model, forecast_input_data)#

Bases: object

add_confidence_interval(forecast, pj)#

Add a confidence interval to a forecast.

Adds a confidence interval to a forecast in two ways:
  1. “stdev” column, this is a column with a standard deviation that is

    determined during training (ConfidenceGenerator)

  2. Quantile columns, these columns give a more precise defenition of the

    confidence interval. Quantile columns are determined with one of two methods, depending on the model type group:

    1. Default, using the “stdev” column and the assumption the error is

      normally distributed.

    2. Quantile regression, this method is only available for quantile

      models and uses specifically trained models to estimate the quantiles of the confidence interval.

    Depending on the model type (quantile or non quantile),

    a confidence interval is added to the forecast based on quantile regression or the default method.

Parameters:

  • forecast (DataFrame) – Forecast DataFrame with columns: “forecast”

  • pj (PredictionJobDataClass) – Prediction job

Return type:

DataFrame

Returns:

Forecast DataFrame with columns; “forecast”, “stdev” and quantile columns.

openstef.model.fallback module#

openstef.model.fallback.generate_fallback(forecast_input, load, fallback_strategy='extreme_day')#

Make a fall back forecast, Set the value of the forecast ‘quality’ column to ‘substituted’.

Currently only fallback_strategy=extreme day is implemented which return historic profile of most extreme day.

Parameters:

  • forecast_input (DataFrame) – dataframe desired for the forecast

  • load (DataFrame) – index=datetime, columns=[‘load’]

  • fallback_strategy (str) – strategy to determine fallback. options: - extreme_day: use daily profile of most extreme day

Return type:

DataFrame

Returns:

Fallback forecast DataFrame with columns; ‘forecast’, ‘quality’

Raises:

  • ValueError if len(load) == 0 –

  • NotImplementedError if fallback_strategy != 'extreme_day' –

openstef.model.model_creator module#

class openstef.model.model_creator.ModelCreator#

Bases: object

Factory object for creating machine learning models.

MODEL_CONSTRUCTORS = {<MLModelType.XGB: 'xgb'>: <class 'openstef.model.regressors.xgb.XGBOpenstfRegressor'>, <MLModelType.LGB: 'lgb'>: <class 'openstef.model.regressors.lgbm.LGBMOpenstfRegressor'>, <MLModelType.XGB_QUANTILE: 'xgb_quantile'>: <class 'openstef.model.regressors.xgb_quantile.XGBQuantileOpenstfRegressor'>, <MLModelType.ProLoaf: 'proloaf'>: None, <MLModelType.LINEAR: 'linear'>: <class 'openstef.model.regressors.linear.LinearOpenstfRegressor'>, <MLModelType.ARIMA: 'arima'>: <class 'openstef.model.regressors.arima.ARIMAOpenstfRegressor'>}#
static create_model(model_type, **kwargs)#

Create a machine learning model based on model type.

Parameters:

  • model_type (Union[MLModelType, str]) – Model type to construct.

  • kwargs – Optional keyword argument to pass to the model.

Raises:

NotImplementedError – When using an invalid model_type.

Return type:

OpenstfRegressor

Returns:

OpenSTEF model

openstef.model.objective module#

class openstef.model.objective.ARIMARegressorObjective(*args, **kwargs)#

Bases: RegressorObjective

get_params(trial)#

Get parameters for ARIMA Regressor Objective with objective specific parameters.

Temporary, it seems strange to use optuna for ARIMA models, it is usually done via statistical analysis and heuristics.

Args: trial

Return type:

dict

Returns:

Dictionary with hyperparameter name as key and hyperparamer value as value.

class openstef.model.objective.LGBRegressorObjective(*args, **kwargs)#

Bases: RegressorObjective

get_params(trial)#

Get parameters for LGB Regressor Objective with objective specific parameters.

Args: trial

Return type:

dict

Returns:

Dictionary with hyperparameter name as key and hyperparamer value as value.

get_pruning_callback(trial)#
class openstef.model.objective.LinearRegressorObjective(*args, **kwargs)#

Bases: RegressorObjective

get_params(trial)#

Get parameters for Linear Regressor Objective with objective specific parameters.

Args: trial

Return type:

dict

Returns:

Dictionary with hyperparameter name as key and hyperparamer value as value.

class openstef.model.objective.ProLoafRegressorObjective(*args, **kwargs)#

Bases: RegressorObjective

get_params(trial)#

Get parameters for ProLoaf Regressor Objective with objective specific parameters.

Args: trial

Return type:

dict

Returns:

Dictionary with hyperparameter name as key and hyperparamer value as value.

get_pruning_callback(trial)#
class openstef.model.objective.RegressorObjective(model, input_data, split_func=None, split_args=None, test_fraction=0.15, validation_fraction=0.15, eval_metric='mae', verbose=False)#

Bases: object

Regressor optuna objective function.

Use any of the derived classes for optimization using an optuna study. The constructor is used to set the “input_data”, specify the splitting function and its arguments and optionally add some configuration. Next the instance will be called by he optuna study during optimization.

Example usage:

# initialize a (derived class) objective function
objective = XGBRegressorObjective(input_data, test_fraction)
# use the objective function
study.optimize(objective)
create_report(model)#

Generate a report from the data available inside the objective function.

Parameters:

model (OpenstfRegressor) – OpenstfRegressor, model to create a report on

Return type:

Report

Returns:

Report about the model

classmethod get_default_values()#
Return type:

dict

get_params(trial)#

Get parameters for objective without model specific get_params function.

Args: trial

Return type:

dict

Returns:

Dictionary with hyperparameter name as key and hyperparamer value as value.

get_pruning_callback(trial)#
get_trial_track()#

Get a dictionary of al trials.

Return type:

dict

Returns:

Dict with al trials and it’s parameters

class openstef.model.objective.XGBQuantileRegressorObjective(*args, **kwargs)#

Bases: RegressorObjective

get_params(trial)#

Get parameters for XGBQuantile Regressor Objective with objective specific parameters.

Args: trial

Return type:

dict

Returns:

Dictionary with hyperparameter name as key and hyperparamer value as value.

get_pruning_callback(trial)#
class openstef.model.objective.XGBRegressorObjective(*args, **kwargs)#

Bases: RegressorObjective

classmethod get_default_values()#
Return type:

dict

get_params(trial)#

Get parameters for XGB Regressor Objective with objective specific parameters.

Args: trial

Return type:

dict

Returns:

Dictionary with hyperparameter name as key and hyperparamer value as value.

get_pruning_callback(trial)#

openstef.model.objective_creator module#

class openstef.model.objective_creator.ObjectiveCreator#

Bases: object

OBJECTIVES = {<MLModelType.XGB: 'xgb'>: <class 'openstef.model.objective.XGBRegressorObjective'>, <MLModelType.LGB: 'lgb'>: <class 'openstef.model.objective.LGBRegressorObjective'>, <MLModelType.XGB_QUANTILE: 'xgb_quantile'>: <class 'openstef.model.objective.XGBQuantileRegressorObjective'>, <MLModelType.ProLoaf: 'proloaf'>: <class 'openstef.model.objective.ProLoafRegressorObjective'>, <MLModelType.LINEAR: 'linear'>: <class 'openstef.model.objective.LinearRegressorObjective'>, <MLModelType.ARIMA: 'arima'>: <class 'openstef.model.objective.ARIMARegressorObjective'>}#
static create_objective(model_type)#

Create an objective function based on model type.

Parameters:

model_type (Union[MLModelType, str]) – Model type to construct.

Raises:

NotImplementedError – When using an invalid model_type.

Return type:

RegressorObjective

Returns:

Objective function

openstef.model.serializer module#

class openstef.model.serializer.MLflowSerializer(mlflow_tracking_uri)#

Bases: object

get_model_age(experiment_name, hyperparameter_optimization_only=False)#

Get model age of most recent model.

Parameters:

  • experiment_name (str) – Name of the experiment, often the id of the predition job.

  • hyperparameter_optimization_only (bool) – Set to true if only hyperparameters optimaisation events should be considered.

Return type:

int

load_model(experiment_name)#

Load sklearn compatible model from MLFlow.

Parameters:

experiment_name (str) – Name of the experiment, often the id of the predition job.

Return type:

tuple[OpenstfRegressor, ModelSpecificationDataClass]

remove_old_models(experiment_name, max_n_models=10, artifact_folder=None)#

Remove old models per experiment.

Note: This functionality is not incorporated in MLFlow natively See also: mlflow/mlflow#2152

save_model(model, experiment_name, model_type, model_specs, report, phase='training', **kwargs)#

Save sklearn compatible model to MLFlow.

Return type:

None

openstef.model.standard_deviation_generator module#

class openstef.model.standard_deviation_generator.StandardDeviationGenerator(validation_data)#

Bases: object

generate_standard_deviation_data(model)#

Generate the standard data.

Calculates the difference between realised and predicted on validation set. For each hour of the day the std of the difference is calculated.

Parameters:

model (RegressorMixin) – The trained model

Return type:

RegressorMixin

Returns:

The model with the std data added.

Module contents#