SemiAnalysis launches GDPx, an agent benchmark on which the best model scores 12%

SemiAnalysis published GDPx on November 2, a benchmark of general agentic capability written by Max Kan, Andrew Megalaa and Dylan Patel. Version 1.0 covers 11 domains: chip design, cybersecurity, biology, finance, legal, quant trading, recursive self-improvement, hardware design, materials science, alignment and software engineering. Unite.AI reports that it holds 206 tasks built with 14 specialized data partners and ranks 13 models. In SemiAnalysis's words, "even the best model scores just 12%."

The leaderboard

Per Unite.AI's summary of the leaderboard, Claude Opus 5.5 ranks first at 12.0%, at about $51.41 per task. Gemini 4 Carbon follows at 10.9% for about $16.29 per task, then GPT-6 Astra at 9.6% for about $53.99, and Meta's Muse Wonder Full at 9.0%. Z.ai's GLM 5.3 places 13th at 2.7%. Headline scores are pass@1 across three attempts, with domains and partners weighted equally.

The lead changes by field. SemiAnalysis calls Opus 5.5 the best all-around model and the leader in quant trading, hardware design and chip design. Gemini 4 Carbon leads biology and narrowly wins software engineering, GPT-6 Astra leads recursive self-improvement and materials science, and Muse Wonder Full is the strongest cyber model by a meaningful margin.

Caveats the authors give

Finance and legal tasks are hard enough that scores are near zero under the intended pass-or-fail grading, so SemiAnalysis reports partial credit for those domains instead. The authors also write that "certain models like Opus 5.5 and GPT-6 Astra have sandbagged overall scores due to their safety classifiers," which affects the cyber, biology and alignment tasks, and that Gemini 4 Carbon's biology lead may partly reflect weaker classifiers. They acknowledge that "any QA system can be gamed given sufficient iteration." Unite.AI reports that in the alignment cyber tasks, 196 of 200 runs broke at least one behavior rule. SemiAnalysis plans new versions every three to six months.

Why it matters

A test where the best score is 12% leaves room to see where agents still fall short on long, expert work, and how far apart the leading models really are. Two findings are directly useful to a buyer. First, no model wins everywhere, so the right choice depends on the work you need done. Second, cost per task varies by more than three times among the top three models, while their scores sit within 2.4 points of each other. If you are choosing a model for agent workloads, run your own representative tasks and compare cost per completed task, which is the number that ends up on the invoice.