ModelPerformanceCallback#

class openstef_models.workflows.callbacks.model_performance_callback.ModelPerformanceCallback(**data: Any) → None[source]

Bases: BaseConfig, ForecastingCallback

Callback for comparing model performance against a treshold during the fit process.

This callback evaluates the model’s performance using a specified metric after fitting. If the performance metric does not meet the defined threshold, a ModelUnderperformingError is raised.

Parameters:

data (Any)

metric_name: str
threshold: float
metric_direction: TypeAliasType
quantile: Union[Quantile, Literal['global']]
on_fit_end(context: WorkflowContext[CustomForecastingWorkflow], result: ModelFitResult) → None[source]

Evaluate model performance at the end of fitting and raise an error if underperforming.

Parameters:

  • context (WorkflowContext[CustomForecastingWorkflow]) – The workflow context that completed fitting.

  • result (ModelFitResult) – Result of the fitting process containing performance metrics.

  • context

  • result

Raises:

ModelUnderperformingError – If the model’s performance metric is below the defined threshold.

Return type:

None

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': False, 'extra': 'ignore', 'protected_namespaces': ()}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_post_init(context: Any, /) → None

This function is meant to behave like a BaseModel method to initialise private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Parameters:

  • self (BaseModel) – The BaseModel instance.

  • context (Any) – The context.

  • self

  • context

Return type:

None