CudaProvider

class openstef_foundation_models.inference.CudaProvider(**data: Any) → None

Bases: BaseConfig

The CUDA (NVIDIA GPU) execution provider.

Parameters:

data (Any)

kind: Literal['cuda']

Discriminator tag for execution-provider type.

device_id: int

CUDA device index to run on.

to_ort() → OrtProvider

Compile to an ONNX Runtime provider tuple.

Return type:

tuple[str, dict[str, object]]

Returns:

The CUDAExecutionProvider with the configured device id.

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': False, 'extra': 'ignore', 'protected_namespaces': ()}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].