← Wednesday, November 18 deck

40Research1 source posting

Bespoke Labs Releases Multi Agent Coordination Bench, and Says Frontier Agents Coordinate Poorly

We’re releasing Multi Agent Coordination Bench, a benchmark of twelve natively multi-agent long-horizon tasks that measure the coordination capabilities of frontier models. We find that current frontier agents coordinate poorly. A team of eight agents beats a single agent, but

Chart from Bespoke Labs showing MAC Bench scores for one agent versus a team of eight, including Opus 5.5, GPT 6.1 Sol, and Gemini 4 Argon.

Context

Why it matters

If frontier agents really gain little from talking to each other on hard team tasks, then adding more agents is not the same thing as adding better coordination.

Bespoke Labs posted that it is releasing Multi Agent Coordination Bench, a benchmark of twelve natively multi agent long horizon tasks, and that current frontier agents coordinate poorly. The on post text is cut off after saying a team of eight agents beats a single agent. A chart attached to the post, described in the company's own alt text, gives scores for Opus 5.5, GPT 6.1 Sol, and Gemini 4 Argon.

Deep dive

Bespoke Labs said it is releasing Multi Agent Coordination Bench, a benchmark of twelve natively multi agent long horizon tasks that measure how well frontier models coordinate. It said current frontier agents coordinate poorly, and that a team of eight agents beats a single agent, but the remainder of that sentence is not in the returned post text.

The company attached a leaderboard image. The official alt text on that image says each bar shows the score of a single agent and the gain from more agents, and that an arrow points to the mean final score of a team of eight. According to that alt text, Opus 5.5 scores 0.534, with a gain from coordination, while GPT 6.1 Sol scores 0.406 and Gemini 4 Argon scores 0.319, each with a loss from coordination.

Those figures are Bespoke's own results, shown on its own chart. This file does not independently run the benchmark. The official benchmarks page lists Multi Agent Coordination Bench as a test of how well teams of agents coordinate on twelve natively multi agent tasks, but the page itself does not republish the full scores in the text fetched here.

A first party benchmark is useful, especially one that tries to separate team size from communication. It is still one lab's test, on a set of tasks and harnesses the post does not fully specify in the remaining text.

What to expect next

A paper, a results table, or a methods note from Bespoke would fill in the cut off sentence and the test setup. Watch whether other labs try the same tasks, and whether newer models change the coordination gap shown for Opus 5.5 versus GPT 6.1 Sol and Gemini 4 Argon. These scores should stay attributed to Bespoke until they are independently checked. This belongs with the day's other benchmark releases, not with a model launch.