TensorRTProvider

class openstef_foundation_models.inference.providers.TensorRTProvider(**data: Any) → None

Bases: BaseConfig

The TensorRT execution provider (NVIDIA, ahead-of-time engine build).

FP16 with a persistent engine cache is the recommended production path on NVIDIA hardware: the first run pays the engine-build cost, subsequent runs load the cached engine.

Parameters:

data (Any)

kind: Literal['tensorrt']

device_id: int

fp16: bool

engine_cache_dir: Path | None

to_ort() → OrtProvider

Compile to an ONNX Runtime provider tuple.

Return type:

tuple[str, dict[str, object]]

Returns:

The TensorrtExecutionProvider with precision and engine-cache options.

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': False, 'extra': 'ignore', 'protected_namespaces': ()}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].