Reference Document

Rolling Dataset Review Checklist

Back To Rolling Submission Submit Options

Rolling Safety

Upload protocol files, not raw audio.

Contributions should describe how to obtain the data, prepare manifests, and construct trials. Maintainers review them separately from the current core ranking.

Do not upload large raw audio. Include download or placement instructions. Use collection="rolling" for accepted catalog patches.

Rolling Dataset Review Checklist

Summary

Dataset ID:
Trial set ID(s):
Proposed scenario or robustness axis:
Public data source:
License / terms:

Reproducibility

[ ] No large raw audio or generated embeddings are committed.
[ ] Download or placement script is included.
[ ] Prepare script is included and safe to rerun.
[ ] Trial construction is deterministic and documents the random seed when sampling is used.
[ ] Trial files use utt1 utt2 label condition.
[ ] All trial utterances exist in wav.scp or wav_all.scp.
[ ] Dataset README includes speaker, utterance, target, nontarget, and condition counts.

Protocol

Protocol source: official / official subset / self-constructed
Source split:
Target rule:
Nontarget rule:
Conditions:
Known limitations:

Core Safety

[ ] Catalog additions use collection="rolling".
[ ] The contribution does not modify official core ranking unless explicitly requested by maintainers.
[ ] python3 tools/generate_submission.py --data <data_root> --precheck-only still passes for the current core.