aiaccel.torch.datasets.FileCachedDataset#

class aiaccel.torch.datasets.FileCachedDataset(dataset: Dataset[T_co], cache_path: str | Path)[source]#

A dataset wrapper that caches samples to disk to reduce memory usage.

This class wraps an existing torch.utils.data.Dataset and caches samples as pickle files in a specified directory.

Parameters:

  • dataset (Dataset[T]) – The dataset to wrap.

  • cache_path (str | Path) – Directory where cached samples will be stored.

__len__()[source]#

Returns the number of samples in the dataset.

__getitem__(index

int) -> Any: Retrieves a sample from cache or the original dataset.

__init__(dataset: Dataset[T_co], cache_path: str | Path) → None[source]#

Methods

__init__(dataset, cache_path)