Benchmark Scenario

In-The-Wild Robustness

How strong the model is on unconstrained celebrity and media speech across official CN-Celeb and VoxCeleb protocols.

In-the-wild Open-domain media speech 4 trial sets

Back To Scenario Index Open Global Leaderboard

Current Leader

wespeaker/w2vbert2

18 ranked

5.1834

scenario EER

0.162046 scenario minDCF

Scenario Coverage

4 trial sets

CN-Celeb VoxCeleb1-O VoxCeleb1-E VoxCeleb1-H

Scenario Leaders

Top systems on this scenario

The current reviewed submissions for this scenario.

wespeaker/w2vbert2

vb2-w2vbert

5.1834

scenario EER

0.162046 scenario minDCF

wespeaker/samresnet100

vb2-sam100

5.2487

scenario EER

0.170274 scenario minDCF

iic/eres2netv2-zh

eresv2_zh

5.5369

scenario EER

0.294964 scenario minDCF

Scenario Ranking

Full ranking for In-The-Wild Robustness

A scenario row first averages linked trial sets within each source dataset, then averages the source means and ranks systems by the resulting EER.

This Scenario All Scenarios Global Board

Rank	System	Rank Signal	Scenario EER	Scenario minDCF	Evidence Rank / EER by trial-set order: CN-Celeb · VoxCeleb1-O · VoxCeleb1-E · VoxCeleb1-H
#1	wespeaker/w2vbert2 vb2-w2vbert	Leader	5.1834	0.162046	#4 9.9690 #1 0.1489 #1 0.3146 #1 0.7298
#2	wespeaker/samresnet100 vb2-sam100	Top 3	5.2487	0.170274	#5 9.9803 #2 0.2287 #3 0.4564 #2 0.8665
#3	iic/eres2netv2-zh eresv2_zh	Top 3	5.5369	0.294964	#1 3.8243 #14 6.3852 #14 6.3659 #14 8.9975
#4	wespeaker/res152-voxceleb wespeaker_resnet152	Competitive	6.1011	0.224787	#6 11.3672 #4 0.4998 #6 0.7174 #5 1.2879
#5	palabraai/redimnet2-b6 redimnet2	Competitive	6.4216	0.208137	#9 12.2444 #3 0.2871 #4 0.5198 #4 0.9892
#6	wespeaker/res34-voxceleb wespeaker_resnet34_voxceleb	Competitive	6.5576	0.240625	#7 11.9718 #8 0.8188 #8 0.9347 #7 1.6768
#7	wespeaker/res293-voxceleb wespeaker_resnet293	Competitive	6.6344	0.231795	#10 12.4334 #5 0.5159 #5 0.7016 #6 1.2890
#8	iic/eres2net-en eres_en	Competitive	6.6673	0.251235	#8 12.1487 #7 0.8028 #9 0.9682 #9 1.7866
#9	iic/campplus campplus	Competitive	6.8211	0.361956	#2 4.7029 #16 7.7917 #16 7.9032 #16 11.1228
#10	wespeaker/res34-cnceleb wespeaker_r34	Needs work	7.8434	0.404734	#3 7.4729 #15 7.0897 #15 7.1685 #15 10.3836
#11	speechbrain/ecapa ecapa	Needs work	7.9977	0.317381	#12 15.2177 #10 0.9038 #2 0.4563 #3 0.9731
#12	wespeaker/campplus-voxceleb campplus_wespeaker	Needs work	8.4036	0.311533	#13 15.7139 #6 0.7180 #7 0.8789 #8 1.6832
#13	wespeaker/ecapa1024-voxceleb wespeaker_ecapa1024	Needs work	8.5204	0.308148	#14 15.8209 #9 0.8188 #10 0.9899 #10 1.8513
#14	wespeaker/ecapa512-voxceleb wespeaker_ecapa512	Needs work	8.9203	0.324306	#15 16.3391 #11 1.0371 #11 1.2104 #11 2.2570
#15	iic/eres2net-large-3dspeaker eres2net_large_3dspeaker	Needs work	14.2563	0.581636	#11 14.0524 #18 13.0837 #18 13.1915 #18 17.1053
#16	microsoft/wavlm-base-plus-sv wavlm_base	Needs work	16.5057	0.632600	#17 27.9358 #12 5.5047 #12 3.2098 #12 6.5121
#17	speechbrain/xvector xvector	Needs work	17.4683	0.675055	#16 24.2749 #17 8.9618 #17 9.1649 #17 13.8588
#18	microsoft/unispeech-sat-base-plus-sv unispeech_sat_base_plus	Needs work	17.6846	0.670598	#18 29.8022 #13 5.6037 #13 3.6382 #13 7.4593