harpy.tb.ZarrDataLoader

harpy.tb.ZarrDataLoader#

class harpy.tb.ZarrDataLoader(*args, start_epoch=0, **kwargs)#

DataLoader that increments epoch and forwards it to epoch-aware datasets.

This wrapper around torch.utils.data.DataLoader maintains an internal epoch counter and, at the start of each iteration, calls set_epoch(epoch) on the underlying dataset when available. This is intended for ZarrIterableInstances (and similar datasets) that need deterministic, epoch-dependent shuffling while also supporting multi-worker and DDP use.

Parameters:

*args – Passed through to torch.utils.data.DataLoader.
**kwargs – Passed through to torch.utils.data.DataLoader.
start_epoch (int (default: 0)) – Initial epoch counter value. The current epoch is applied on the first call to __iter__ and then incremented after the iterator is created.

Notes

The current epoch can be read via _get_epoch(). This class does not override any other DataLoader behavior.

Attributes table#

`multiprocessing_context`
`dataset`
`batch_size`
`num_workers`
`pin_memory`
`drop_last`
`timeout`
`sampler`
`pin_memory_device`
`prefetch_factor`

Methods table#

check_worker_number_rationality()

Attributes#

ZarrDataLoader.multiprocessing_context#

ZarrDataLoader.dataset: Dataset[_T_co]#

ZarrDataLoader.batch_size: int | None#

ZarrDataLoader.num_workers: int#

ZarrDataLoader.pin_memory: bool#

ZarrDataLoader.drop_last: bool#

ZarrDataLoader.timeout: float#

ZarrDataLoader.sampler: Sampler | Iterable#

ZarrDataLoader.pin_memory_device: str#

ZarrDataLoader.prefetch_factor: int | None#

Methods#

ZarrDataLoader.check_worker_number_rationality()#

Return type:: None