Canonical dataset contract¶

Maturity: verified

One canonical bundle has this exact file layout:

<bundle-root>/
  manifest.json
  metadata.parquet
  signals.h5
  splits/
    train.parquet
    val.parquet
    test.parquet

HDF5 datasets¶

Name	Required	Contract
`data`	yes	float32-compatible rows shaped `(N, 2, T)`, I then Q
`sample_ids`	yes	unique string identity aligned one-to-one with `data` rows
`labels`	dataset-dependent	source labels when present
`snrs`	dataset-dependent	source signal-to-noise ratio values when present

The training reader joins signal rows to metadata and split membership through sample_ids; it does not assume that split order equals HDF5 row order.

Metadata and splits¶

metadata.parquet has one unique row per canonical sample ID. Common columns include sample_id, optional label, optional snr_db, and split. Dataset-specific columns remain in the sample’s metadata attributes.

Each split Parquet contains ordered, unique sample_id membership. Every split ID must exist in metadata and HDF5. One reader instance selects exactly one of train, val, or test.

Manifest¶

The shared validator requires:

dataset name, data-contract identity, dataset snapshot identity;
source URL, citation, ingestion date, and preprocessing identity;
canonical shape, sample rate, optional center frequency and bandwidth;
split group columns, proportions, seed, and split identity;
signal-to-noise-ratio dtype;
provenance mapping and artifact checksum mapping.

The snapshot ID and checksums identify concrete content. The current data-contract, preprocessing, and split fields are existing external artifact vocabulary consumed by the reader and registry; public docs do not invent a new compatibility version.

Reader failures¶

Construction fails on missing artifacts, invalid manifests, duplicate or empty sample IDs, or membership that cannot be joined across files. Sample creation fails on invalid I/Q shape, dtype, or provenance.