Datasets API Reference

Dataset Factories

SkiNet.ML.datasets.dataset_factory.create_segmentation_datasets_from_config(config: ExperimentConfig) → DatasetSplit[SegmentationDataset]

Typed entry point for segmentation experiments.

Example:

dataset_splits = create_segmentation_datasets_from_config(config)
loader = DataLoader(dataset_splits.train, batch_size=8, shuffle=True)

Parameters:

config – Experiment configuration for a segmentation experiment.

Returns:

A DatasetSplit[SegmentationDataset] with typed train/val/test splits.

class SkiNet.ML.datasets.dataset_factory.DatasetSplit(train: TDataset_co, val: TDataset_co, test: TDataset_co, splits: DataFrameSplits)

Bases: Generic[TDataset_co]

Group of train/validation/test datasets created from one split operation.

The container is generic so orchestration code can use the common shape while experiment-specific code can preserve concrete dataset types

train

Dataset for model training.

Type:

SkiNet.ML.utils.typing_utils.TDataset_co

val

Dataset for validation and early stopping.

Type:

SkiNet.ML.utils.typing_utils.TDataset_co

test

Dataset for final held-out evaluation.

Type:

SkiNet.ML.utils.typing_utils.TDataset_co

splits

Raw dataframe splits used to construct the train, val, test datasets.

Type:

SkiNet.Utils.data.split_data.DataFrameSplits

splits: DataFrameSplits

test: TDataset_co

train: TDataset_co

val: TDataset_co

class SkiNet.ML.datasets.dataset_factory.DatasetFactory

Bases: ABC, Generic[TDataset_co]

Base class for experiment-specific dataset factories.

A concrete factory is responsible for deriving the split dataframes, selecting the transforms for each workflow stage, and returning a typed DatasetSplit for the requested experiment type.

abstractmethod create_datasets(config: ExperimentConfig) → DatasetSplit[TDataset_co]

Build the full train/val/test dataset container for config.

Implementations are responsible for: - splitting the source metadata into train/val/test dataframes - resolving the appropriate transforms per split - constructing and returning a typed DatasetSplit

Parameters:

config – Fully resolved experiment configuration.

Returns:

A DatasetSplit typed to this factory’s dataset type.

class SkiNet.ML.datasets.dataset_factory.SegmentationDatasetFactory

Bases: DatasetFactory[SegmentationDataset]

Factory that creates SegmentationDataset objects for each workflow split. Use create_segmentation_datasets_from_config for the typed public entry point.

create_datasets(config: ExperimentConfig) → DatasetSplit[SegmentationDataset]

Create the segmentation train/validation/test datasets for config.

Parameters:

config – Experiment configuration containing segmentation metadata, split configuration, transform configuration, and data root.

Returns:

A DatasetSplit[SegmentationDataset] containing typed train, val, and test splits alongside the raw DataFrameSplits used to construct them.

---

Datasets

class SkiNet.ML.datasets.segmentation_dataset.BaseDataset(*args: Any, **kwargs: Any)

Bases: Dataset, ABC

class SkiNet.ML.datasets.segmentation_dataset.SegmentationDataset(*args: Any, **kwargs: Any)

Bases: BaseDataset

Dataset for semantic segmentation tasks.

data_root

Data root path where images and masks are stored, derived from the experiment configuration.

dataframe

A pandas DataFrame containing metadata for the dataset. It should be provided directly for train, val and test modes of operation after deriving it as a respective subset of the full dataframe.

get_raw_sample(index: int) → Sample

Load a raw sample from disk without applying any transforms. Useful for visualization and debugging without mutating dataset.transform.

get_sample_item(index: int) → dict[str, Any]

Get a single sample item by index.

Returns:

A dictionary containing the image tensor, mask tensor, and sample specifications for the specified index.

sample_ids

A list of sample IDs corresponding to the valid samples in the dataset, derived from the sample specifications.

sample_specs

A dictionary containing the valid sample specifications, derived from the DataFrame, such as image and mask paths and metadata.

---

Supported Experiment Types

ExperimentType	Factory	Dataset class
SEGMENTATION	SegmentationDatasetFactory	SegmentationDataset

---

Extending

To support a new experiment type, subclass DatasetFactory and register it:

class ClassificationDatasetFactory(DatasetFactory):
    def create_datasets(self, config: ExperimentConfig) -> DatasetSplit:
        ...

dataset_factories = {
    ExperimentType.SEGMENTATION:   SegmentationDatasetFactory(),
    ExperimentType.CLASSIFICATION: ClassificationDatasetFactory(),
}

---

Internals

create_datasets() runs three steps in order:

1. Split — split_segmentation_metadata() partitions the metadata DataFrame into train/val/test subsets.

2. Transform — get_transform_from_config() builds mode-specific augmentation pipelines.

3. Construct — one SegmentationDataset is instantiated per split, each receiving its corresponding dataframe, transform branch, and MLWorkflowState mode.