Contra Labs Launches the Frontier Design Benchmark, Ranking AI Models on Real Client Design Briefs
The AI industry has a benchmarking problem. We're letting random people on the internet decide which models are best even for capabilities like design... So we built something different. Introducing The Frontier Design Benchmark from @contralabs
Context
A design benchmark judged by professional designers on paid client work gives builders a sturdier signal than crowd votes when choosing a model for websites and marketing pages.
Ben, posting for Contra Labs, introduced The Frontier Design Benchmark, saying the AI industry has a benchmarking problem because random people online decide which models are best. The benchmark site says it is built from real client briefs on Contra's marketplace and judged by professional designers in the Contra Labs network. It is a joint project of Contra Labs and BenchFlow.
Deep dive
Ben of Contra Labs posted that the AI industry has a benchmarking problem, arguing that letting random people on the internet decide which models are best falls short even for capabilities like design. He introduced The Frontier Design Benchmark from Contra Labs as the alternative, with a launch video and a link to the benchmark site.
According to the site, the benchmark is built on real, high value and complex design projects taken from client briefs on Contra's marketplace. Professional designers in the Contra Labs network judge the outputs in blind head to head votes, and each brief has a golden design made by a person, shown next to the model outputs as the bar. The site lists 33 design briefs, 68 professional designers, 4,311 blind votes and 35 rubric dimensions. It also describes a design critique rubric and a list of AI slop patterns, and says the rubric scores correlate with the designers' ratings at r = 0.94.
On the Elo leaderboard shown on the site, an entry called Gestalt ranks first and the human golden designs rank second. Among named frontier models, Claude Fable 5.2 leads, followed by Claude Opus 5.5, GPT 6.1 Sol and Claude Sonnet 5.5. Kimi K3.1 Pro, GPT 6 Astra and Qwen3.8 Max come next, while Gemini 4 Carbon sits 12th and Grok 4.7 15th. The list does not include Gemini 4 Neon, which Google released later the same day.
These are Contra Labs' own results on its own task set. The site also invites visitors to vote on outputs, so the published ranking may shift as votes are added.
What to expect next
Watch for Contra Labs to add newer models such as Gemini 4 Neon and to publish more of its methodology and dataset, which the site links. Independent designers and model labs may test whether the rubric really tracks professional judgment beyond these 33 briefs. If labs start citing the benchmark in launch posts, it could become a common reference for design work alongside coding and agent benchmarks. Open questions include what the top entry, Gestalt, is and how often the leaderboard will be refreshed.
