Benchmarks people trust to tell if an LLM is frontier: Frontier List

Votes	Benchmark		Area
25	Terminal-Bench 4	Alex ShawS., Andy KonwinskiK., Mike A. MerrillM.	Coding
20	Terminal-Bench Science	Steven DillmannD., Ludwig SchmidtS., Sanmi KoyejoK.	Science
13	FrontierCode	Eric LuL., Ben PanP., Deniz BirlikciB., Silas AlbertiA.	Coding
10	FrontierSWE	Evan ChuC., Rajan AgarwalA., Justus MatternM., Rishyanth KondraK., Sanket MhatreM., Akshit KumarK.	Coding
8	Humanity's Last Exam	Long PhanP., Dan HendrycksH., Alice GattiG., Ziwen HanH., Nathaniel LiL.	Expert knowledge
7	Vending-Bench	Axel BacklundB., Lukas PeterssonP.	Business
5	AutomationBench	Daniel ShepardS., Robin SalimansS., Anna Marie CliftonC.	Business
5	DeepSWE	Wenqi (Kevin) HuangH., Charley LeeL., Leonard TngT., Serena GeG.	Coding
5	PostTrainBench	Ben RankR., Hardik BhatnagarB., Ameya PrabhuP., Shira EisenbergE., Karina NguyenN., Matthias BethgeB., Maksym AndriushchenkoA.	ML engineering
4	AA-Briefcase	Micah Hill-SmithH., George CameronC.	Knowledge work
4	Agents' Last Exam	Yiyou SunS., Dawn SongS., Xinyang HanH., Weichen ZhangZ.	Knowledge work
4	FrontierCS 2	Qiuyang MangM., Wenhao ChaiC., Runyuan HeH., Dongruixuan LiL., Zihan ZhengZ., Yujun HeH., Alvin CheungC., Joseph E. GonzalezG.	Coding
4	OSWorld 2	Mengqi YuanY., Zilong ZhouZ., Xinzhuang XiongX., Tao YuY., Tianbao XieX.	Computer use
3	ARC-AGI-3	Greg KamradtK., François CholletC., Mike KnoopK., Hunter HenryH.	Reasoning
3	FrontierMath	Elliot GlazerG., Tamay BesirogluB., Ege ErdilE.	Math
3	Harvey LAB	Niko GrupenG., Gabe PereyraP., Julio PereyraP., Spencer PoffP.	Legal
3	ProgramBench	John YangY., Kilian LieretL., Jeffrey Jian MaM., Diyi YangY., Ofir PressP.	Coding
3	SWE-sweep	Kilian LieretL., Jeffrey Jian MaM., Ofir PressP., John YangY.	Coding
2	GDPx	Max KanK., Andrew MegalaaM., Dylan PatelP., Abhijay RanaR.	Multi-domain
2	BrowseComp	Jason WeiW., Zhiqing SunS., Hyung Won ChungC., Mia (Amelia) GlaeseG.	Web research
2	Chartography	Suhaas GarreG., Chris MuttyM., Sushant MehtaM., Edwin ChenC.	Visual reasoning
2	CursorBench	Naman JainJ.	Coding
2	ExploitBench	Seunghyun Lee (Xion)(., David BrumleyB.	Cyber
2	GDP.pdf	Suhaas GarreG., Emily RitchieR., Sushant MehtaM., Edwin ChenC.	Knowledge work
2	GDPval-AA	Tejal PatwardhanP., Micah Hill-SmithH., George CameronC., Declan JacksonJ.	Knowledge work

Counts are net: a few benchmarks got negative votes.

GDPx came out the day before I sent the survey.

GDPx was built with 14 data partners: Arcverex, Aria, Arithmetic, Beautiful Machines, Chakra, Datacurve, Emulated, Ergodex, Extrapolate, Fleet, Latch, Living Context, Third Moment, and ValenceForge.

The most common comment was that people don't really trust benchmarks at all.