DatasetMixin#
- class openstef_core.datasets.mixins.DatasetMixin(*args, **kwargs)[source]
Bases:
ProtocolAbstract base class for dataset persistence operations.
This mixin defines the interface for saving and loading datasets to/from parquet files. It ensures datasets can be persisted with all their metadata and reconstructed exactly as they were saved.
Classes implementing this mixin must:
Save all data and metadata necessary for complete reconstruction
Store metadata in parquet file attributes using attrs
Handle missing metadata gracefully with sensible defaults when loading
See also
TimeSeriesDataset: Implementation for standard time series datasets. VersionedTimeSeriesDataset: Implementation for versioned dataset segments.
- abstractmethod to_parquet(path: Path) None[source]
Save the dataset to a parquet file.
Stores both the dataset’s data and all necessary metadata for complete reconstruction. Metadata should be stored in the parquet file’s attrs dictionary.
See also
read_parquet: Counterpart method for loading datasets.
- abstractmethod classmethod read_parquet(path: Path) Self[source]
Load a dataset from a parquet file.
Reconstructs a dataset from a parquet file created with to_parquet, including all data and metadata. Should handle missing metadata gracefully with sensible defaults.
- Parameters:
path (
Path) – Path to the parquet file to load.- Return type:
- Returns:
New dataset instance reconstructed from the file.
See also
to_parquet: Counterpart method for saving datasets.
- pipe[T, **P](func: Callable[Concatenate[Self, P], T], *args: P.args, **kwargs: P.kwargs) T[source]
Applies a function to the dataset and returns the result.
This method allows for functional-style transformations and operations on the dataset, enabling method chaining and cleaner code.
- Parameters:
func (
Callable[Concatenate[Self,P],T]) – A callable that takes the dataset instance and returns a value of type T.*args (
P.args) – Positional arguments to pass to the function.**kwargs (
P.kwargs) – Keyword arguments to pass to the function.
- Return type:
T- Returns:
The result of applying the function to the dataset.
- __init__(*args, **kwargs)