Deepnote's Agent Initiative Benchmark Tests 12 Models on When to Help at Work, With GPT 6 Astra on Top
What’s the best model for proactive agents? Today, we’re introducing the Agent Initiative Benchmark. We tested 12 frontier models on realistic workplace tasks inside a synthetic company with 6 years of highly detailed history: it has a financial model, CRM records, past-due
Context
It measures a skill always on agents need but few tests cover: deciding whether to speak up at all, and staying quiet when help is not needed.
Deepnote founder Jakub Jurovych introduced the Agent Initiative Benchmark on November 13, and Deepnote published the results and an open source repository under the Apache 2.0 license. The test is set inside PageMantle, a synthetic software company with six years of history spread across 11 connected services.
Deep dive
Deepnote says each of 12 models faced the same 50 workplace situations across eight task types, from finding facts to checking whether an action follows company rules. Every model got identical instructions, tools and company data, ran at high reasoning effort, and had a budget of 180 seconds of model response time. The agent reads recent Slack messages, can search and read across 11 services, including running SQL on BigQuery and Snowflake, and then posts in a channel, replies in a thread, sends a direct message or stays silent. Two LLM evaluators that do not know which model they are grading score each run from 0 to 10 against a reference solution.
On Deepnote's leaderboard, GPT 6 Astra scored 8.97, GPT 6.1 Sol 8.65 and Claude Fable 5.2 8.17, followed by Claude Opus 5.5 at 7.84. Gemini 4 Argon scored 5.60 and Grok 4.7 finished last at 4.51. Cost per task ranged from $0.007 for GPT 6 Luna to $3.02 for Claude Fable 5.1.
Deepnote says models struggled most with restraint. According to Agentic Feed's summary, when help was needed a message went out nine times in ten, but when silence was right models held back only six times in ten. Deepnote lists limits of its own: only 50 tasks, a single effort level, and a design that compares models rather than harnesses.
What to expect next
Deepnote says it is building a harness benchmark and plans a larger task set in the next version, which will show whether the ranking holds when prompts and tool policies change. Watch for labs or independent groups to reproduce the results with the open source repository, and for scores from models not yet tested. Because two LLM judges grade every run, comparisons with human grading would help show how reliable the scores are.