Diagnostic Index

Benchmark scenarios

Each scenario groups related trial sets around a deployment condition and provides a local ranking for diagnosing system behavior.

11 scenarios 26 trial sets 25 ranked systems

Global leaderboard Trial-set index

Scenario Rankings

Evaluation coverage

Open a scenario to compare systems using the corresponding group-level EER, minDCF, and trial-set evidence.

Scenario	Evaluation Focus	Trial Sets	Included Trial Sets	Current Leader
Accent / Dialect Robustness Accent/dialect	Accent and dialect variation	2	GLOBE 3D-Speaker Dialect	w2v-bert2.0 + cc + ecapa-tdnn + astp + sf2 JinghanPeng
Aging Robustness Aging	Speaker aging and time gaps	2	VoxKnesset VoxPopuli Aging	W2V-BERT 2.0 + MHFA (VoxCeleb2 + VoxBlink2) pengjy
Channel / Device Robustness Channel/device	Device and channel variation	2	3D-Speaker Device FFSVC 2022 Cross-Channel	MECT-b2-czur-finetuned czur
Cross-Lingual Robustness Cross-lingual	Language mismatch	1	TidyVoiceX2-ASV	Dual-LoRA W2V-BERT2-Adapter-MFA (TidyVoice) sgqt
Distance Robustness Distance	Distance mismatch	4	3D-Speaker Distance AliMeeting Near/Far CHiME-6 Domestic Far-Field +1	w2v-bert2.0 + cc + ecapa-tdnn + astp + sf2 JinghanPeng
Genre-Shift Robustness Genre shift	Source-genre variation	1	CN-Celeb Genre	w2v-bert2.0 + cc + ecapa-tdnn + astp + sf2 JinghanPeng
In-The-Wild Robustness In-the-wild	Open-domain media speech	4	CN-Celeb VoxCeleb1-O VoxCeleb1-E +1	w2v-bert2.0 + cc + ecapa-tdnn + astp + sf2 JinghanPeng
Noise / Reverb Robustness Noise/reverb	Noise and reverberation	1	VOiCES Noise/Reverb	palabraai/redimnet2-b6-vb2+vox2 Public checkpoint
Overlap Robustness Overlap	Overlapping speakers	2	AliMeeting Overlap CHiME-6 Overlap	w2v-bert2.0 + cc + ecapa-tdnn + astp + sf2 JinghanPeng
Short-Duration Robustness Short-duration	Short-duration speech	4	HI-MIA GSC Short CN-Celeb Short +1	w2v-bert2.0 + cc + ecapa-tdnn + astp + sf2 JinghanPeng
Speaking-Style Robustness Speaking style	Speaking style shift	3	ESD Whisper40 Whisper Lombard Grid Lombard	W2V-BERT 2.0 + MHFA (VoxCeleb2 + VoxBlink2) pengjy