ProviderPolicy#

class openstef_foundation_models.inference.provider_selection.ProviderPolicy(*args, **kwargs)[source]

Bases: Protocol

Port mapping a checkpoint and host to an ordered execution-provider chain.

Implement this to encode selection rules for hardware the default policy does not cover; pass the implementation to the backend or OnnxBackendConfig.

A policy-selected chain is enforced gracefully: ONNX Runtime silently drops accelerators it cannot initialize and falls back to CPU, and a policy chain such as [CoreML, CPU] realizing CoreML is the intended outcome, so a warning is logged only if it falls all the way to CPU. A chain the caller passes explicitly is enforced strictly instead: any requested accelerator that is not realized raises.

select(metadata: CheckpointMetadata, host: HostCapabilities) → list[Annotated[CpuProvider | CudaProvider | TensorRTProvider | CoreMLProvider, FieldInfo(annotation=NoneType, required=True, discriminator='kind')]][source]

Return the ordered provider chain to try for metadata on host.

Parameters:

  • metadata (CheckpointMetadata)

  • host (HostCapabilities)

Return type:

list[CpuProvider | CudaProvider | TensorRTProvider | CoreMLProvider]

__init__(*args, **kwargs)