DefaultProviderPolicy#

class openstef_foundation_models.inference.provider_selection.DefaultProviderPolicy(*, kind: Literal['default'] = 'default')[source]

Bases: BaseConfig

Default policy mapping (checkpoint precision/shape, host) to a provider chain.

Each rule encodes a measured hardware conclusion; see the design doc design-docs/0001 and the provider benchmark for the rationale. The chain is ordered preferred-first with CPU as the final fallback.

Parameters:

kind (Literal['default'])

kind: Literal['default']
select(metadata: CheckpointMetadata, host: HostCapabilities) → list[CpuProvider | CudaProvider | TensorRTProvider | CoreMLProvider][source]

Select an ordered provider chain for metadata on host.

Parameters:
  • metadata (CheckpointMetadata) – The checkpoint’s metadata (precision, static-shape-ness).

  • host (HostCapabilities) – The detected host capabilities.

Return type:

list[CpuProvider | CudaProvider | TensorRTProvider | CoreMLProvider]

Returns:

An ordered execution-provider chain, preferred-first, CPU last.

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': False, 'extra': 'ignore', 'protected_namespaces': ()}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].