openstef.model package#
Subpackages#
- openstef.model.metamodels package
- Submodules
- openstef.model.metamodels.grouped_regressor module
- openstef.model.metamodels.missing_values_handler module
MissingValuesHandler
MissingValuesHandler.feature_names
MissingValuesHandler.non_null_columns_
MissingValuesHandler.n_features_in_
MissingValuesHandler.regressor_
MissingValuesHandler.imputer_
MissingValuesHandler.pipeline_
MissingValuesHandler.feature_importances_
MissingValuesHandler.fit()
MissingValuesHandler.predict()
- Module contents
- openstef.model.regressors package
- Submodules
- openstef.model.regressors.arima module
ARIMAOpenstfRegressor
ARIMAOpenstfRegressor.can_predict_quantiles
ARIMAOpenstfRegressor.feature_names
ARIMAOpenstfRegressor.fit()
ARIMAOpenstfRegressor.predict()
ARIMAOpenstfRegressor.predict_quantile()
ARIMAOpenstfRegressor.score()
ARIMAOpenstfRegressor.set_feature_importance()
ARIMAOpenstfRegressor.update_historic_data()
- openstef.model.regressors.custom_regressor module
- openstef.model.regressors.dazls module
- openstef.model.regressors.lgbm module
- openstef.model.regressors.linear module
- openstef.model.regressors.proloaf module
- openstef.model.regressors.regressor module
- openstef.model.regressors.xgb module
- openstef.model.regressors.xgb_quantile module
- Module contents
Submodules#
openstef.model.basecase module#
- class openstef.model.basecase.BaseCaseModel#
Bases:
BaseEstimator
,RegressorMixin
- property can_predict_quantiles#
- fit()#
- static make_basecase_forecast(forecast_input_data, overwrite_delay_hours=48)#
Make a basecase forecast.
The idea of the basecase forecast is that if all else fails, this forecasts is still available. Basecase example: the load of last week.
- Parameters:
forecast_input_data (
DataFrame
) – Forecast input dataframeoverwrite_delay_hours (
int
) – times before this in the future are not forecasted
- Raises:
ValueError – if columns T-7d or T-14d is not present
ValueError – If the start of the forecast is before the horizon of the regular forecast
- Return type:
DataFrame
- Returns:
Basecase forecast
- predict(forecast_input_data)#
Predict using the basecase method. The basecase forecast is determined by the T-7d and T-14d load.
This means fitting the model is not required. However a fit method is still included to be fully comatible with sklearn.
- Parameters:
forecast_input_data (
DataFrame
) – Forecast input dataframe- Return type:
DataFrame
- Returns:
Basecase forecast
openstef.model.confidence_interval_applicator module#
- class openstef.model.confidence_interval_applicator.ConfidenceIntervalApplicator(model, forecast_input_data)#
Bases:
object
- add_confidence_interval(forecast, pj)#
Add a confidence interval to a forecast.
- Adds a confidence interval to a forecast in two ways:
- “stdev” column, this is a column with a standard deviation that is
determined during training (ConfidenceGenerator)
- Quantile columns, these columns give a more precise defenition of the
confidence interval. Quantile columns are determined with one of two methods, depending on the model type group:
- Default, using the “stdev” column and the assumption the error is
normally distributed.
- Quantile regression, this method is only available for quantile
models and uses specifically trained models to estimate the quantiles of the confidence interval.
- Depending on the model type (quantile or non quantile),
a confidence interval is added to the forecast based on quantile regression or the default method.
- Parameters:
forecast (
DataFrame
) – Forecast DataFrame with columns: “forecast”pj (
PredictionJobDataClass
) – Prediction job
- Return type:
DataFrame
- Returns:
Forecast DataFrame with columns; “forecast”, “stdev” and quantile columns.
openstef.model.fallback module#
- openstef.model.fallback.generate_fallback(forecast_input, load, fallback_strategy='extreme_day')#
Make a fall back forecast, Set the value of the forecast ‘quality’ column to ‘substituted’.
Currently only fallback_strategy=extreme day is implemented which return historic profile of most extreme day.
- Parameters:
forecast_input (
DataFrame
) – dataframe desired for the forecastload (
DataFrame
) – index=datetime, columns=[‘load’]fallback_strategy (
str
) – strategy to determine fallback. options: - extreme_day: use daily profile of most extreme day
- Return type:
DataFrame
- Returns:
Fallback forecast DataFrame with columns; ‘forecast’, ‘quality’
- Raises:
ValueError if len(load) == 0 –
NotImplementedError if fallback_strategy != 'extreme_day' –
openstef.model.model_creator module#
- class openstef.model.model_creator.ModelCreator#
Bases:
object
Factory object for creating machine learning models.
- MODEL_CONSTRUCTORS = {<MLModelType.XGB: 'xgb'>: <class 'openstef.model.regressors.xgb.XGBOpenstfRegressor'>, <MLModelType.LGB: 'lgb'>: <class 'openstef.model.regressors.lgbm.LGBMOpenstfRegressor'>, <MLModelType.XGB_QUANTILE: 'xgb_quantile'>: <class 'openstef.model.regressors.xgb_quantile.XGBQuantileOpenstfRegressor'>, <MLModelType.ProLoaf: 'proloaf'>: None, <MLModelType.LINEAR: 'linear'>: <class 'openstef.model.regressors.linear.LinearOpenstfRegressor'>, <MLModelType.ARIMA: 'arima'>: <class 'openstef.model.regressors.arima.ARIMAOpenstfRegressor'>}#
- static create_model(model_type, **kwargs)#
Create a machine learning model based on model type.
- Parameters:
model_type (
Union
[MLModelType
,str
]) – Model type to construct.kwargs – Optional keyword argument to pass to the model.
- Raises:
NotImplementedError – When using an invalid model_type.
- Return type:
- Returns:
OpenSTEF model
openstef.model.objective module#
- class openstef.model.objective.ARIMARegressorObjective(*args, **kwargs)#
Bases:
RegressorObjective
- get_params(trial)#
Get parameters for ARIMA Regressor Objective with objective specific parameters.
Temporary, it seems strange to use optuna for ARIMA models, it is usually done via statistical analysis and heuristics.
Args: trial
- Return type:
dict
- Returns:
Dictionary with hyperparameter name as key and hyperparamer value as value.
- class openstef.model.objective.LGBRegressorObjective(*args, **kwargs)#
Bases:
RegressorObjective
- get_params(trial)#
Get parameters for LGB Regressor Objective with objective specific parameters.
Args: trial
- Return type:
dict
- Returns:
Dictionary with hyperparameter name as key and hyperparamer value as value.
- get_pruning_callback(trial)#
- class openstef.model.objective.LinearRegressorObjective(*args, **kwargs)#
Bases:
RegressorObjective
- get_params(trial)#
Get parameters for Linear Regressor Objective with objective specific parameters.
Args: trial
- Return type:
dict
- Returns:
Dictionary with hyperparameter name as key and hyperparamer value as value.
- class openstef.model.objective.ProLoafRegressorObjective(*args, **kwargs)#
Bases:
RegressorObjective
- get_params(trial)#
Get parameters for ProLoaf Regressor Objective with objective specific parameters.
Args: trial
- Return type:
dict
- Returns:
Dictionary with hyperparameter name as key and hyperparamer value as value.
- get_pruning_callback(trial)#
- class openstef.model.objective.RegressorObjective(model, input_data, split_func=None, split_args=None, test_fraction=0.15, validation_fraction=0.15, eval_metric='mae', verbose=False)#
Bases:
object
Regressor optuna objective function.
Use any of the derived classes for optimization using an optuna study. The constructor is used to set the “input_data”, specify the splitting function and its arguments and optionally add some configuration. Next the instance will be called by he optuna study during optimization.
Example usage:
# initialize a (derived class) objective function objective = XGBRegressorObjective(input_data, test_fraction) # use the objective function study.optimize(objective)
- create_report(model)#
Generate a report from the data available inside the objective function.
- Parameters:
model (
OpenstfRegressor
) – OpenstfRegressor, model to create a report on- Return type:
- Returns:
Report about the model
- classmethod get_default_values()#
- Return type:
dict
- get_params(trial)#
Get parameters for objective without model specific get_params function.
Args: trial
- Return type:
dict
- Returns:
Dictionary with hyperparameter name as key and hyperparamer value as value.
- get_pruning_callback(trial)#
- get_trial_track()#
Get a dictionary of al trials.
- Return type:
dict
- Returns:
Dict with al trials and it’s parameters
- class openstef.model.objective.XGBQuantileRegressorObjective(*args, **kwargs)#
Bases:
RegressorObjective
- get_params(trial)#
Get parameters for XGBQuantile Regressor Objective with objective specific parameters.
Args: trial
- Return type:
dict
- Returns:
Dictionary with hyperparameter name as key and hyperparamer value as value.
- get_pruning_callback(trial)#
- class openstef.model.objective.XGBRegressorObjective(*args, **kwargs)#
Bases:
RegressorObjective
- classmethod get_default_values()#
- Return type:
dict
- get_params(trial)#
Get parameters for XGB Regressor Objective with objective specific parameters.
Args: trial
- Return type:
dict
- Returns:
Dictionary with hyperparameter name as key and hyperparamer value as value.
- get_pruning_callback(trial)#
openstef.model.objective_creator module#
- class openstef.model.objective_creator.ObjectiveCreator#
Bases:
object
- OBJECTIVES = {<MLModelType.XGB: 'xgb'>: <class 'openstef.model.objective.XGBRegressorObjective'>, <MLModelType.LGB: 'lgb'>: <class 'openstef.model.objective.LGBRegressorObjective'>, <MLModelType.XGB_QUANTILE: 'xgb_quantile'>: <class 'openstef.model.objective.XGBQuantileRegressorObjective'>, <MLModelType.ProLoaf: 'proloaf'>: <class 'openstef.model.objective.ProLoafRegressorObjective'>, <MLModelType.LINEAR: 'linear'>: <class 'openstef.model.objective.LinearRegressorObjective'>, <MLModelType.ARIMA: 'arima'>: <class 'openstef.model.objective.ARIMARegressorObjective'>}#
- static create_objective(model_type)#
Create an objective function based on model type.
- Parameters:
model_type (
Union
[MLModelType
,str
]) – Model type to construct.- Raises:
NotImplementedError – When using an invalid model_type.
- Return type:
- Returns:
Objective function
openstef.model.serializer module#
- class openstef.model.serializer.MLflowSerializer(mlflow_tracking_uri)#
Bases:
object
- get_model_age(experiment_name, hyperparameter_optimization_only=False)#
Get model age of most recent model.
- Parameters:
experiment_name (
str
) – Name of the experiment, often the id of the predition job.hyperparameter_optimization_only (
bool
) – Set to true if only hyperparameters optimaisation events should be considered.
- Return type:
int
- load_model(experiment_name)#
Load sklearn compatible model from MLFlow.
- Parameters:
experiment_name (
str
) – Name of the experiment, often the id of the predition job.- Return type:
- remove_old_models(experiment_name, max_n_models=10, artifact_folder=None)#
Remove old models per experiment.
Note: This functionality is not incorporated in MLFlow natively See also: mlflow/mlflow#2152
- save_model(model, experiment_name, model_type, model_specs, report, phase='training', **kwargs)#
Save sklearn compatible model to MLFlow.
- Return type:
None
openstef.model.standard_deviation_generator module#
- class openstef.model.standard_deviation_generator.StandardDeviationGenerator(validation_data)#
Bases:
object
- generate_standard_deviation_data(model)#
Generate the standard data.
Calculates the difference between realised and predicted on validation set. For each hour of the day the std of the difference is calculated.
- Parameters:
model (
RegressorMixin
) – The trained model- Return type:
RegressorMixin
- Returns:
The model with the std data added.