DefaultProviderPolicy#
- class openstef_foundation_models.inference.provider_selection.DefaultProviderPolicy(*, kind: Literal['default'] = 'default')[source]
Bases:
BaseConfigDefault policy mapping
(checkpoint precision/shape, host)to a provider chain.Each rule encodes a measured hardware conclusion; see the design doc
design-docs/0001and the provider benchmark for the rationale. The chain is ordered preferred-first with CPU as the final fallback.- Parameters:
kind (
Literal['default'])
- kind: Literal['default']
- select(metadata: CheckpointMetadata, host: HostCapabilities) list[CpuProvider | CudaProvider | TensorRTProvider | CoreMLProvider][source]
Select an ordered provider chain for metadata on host.
- Parameters:
metadata (
CheckpointMetadata) – The checkpoint’s metadata (precision, static-shape-ness).host (
HostCapabilities) – The detected host capabilities.
- Return type:
list[CpuProvider|CudaProvider|TensorRTProvider|CoreMLProvider]- Returns:
An ordered execution-provider chain, preferred-first, CPU last.
- model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': False, 'extra': 'ignore', 'protected_namespaces': ()}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].