ADEMark v1 Compares 15 Agent Development Environments Against a Bare CLI on One Mac
ADEMark v1 is out. Same Mac mini, same tasks, same model, bare CLI as the baseline. Did not expect most of them to use fewer tokens than the CLI on its own. Or one to sit at 16% CPU doing nothing. First of many, updates come way more often from here.
Context
As agent coding apps multiply, a shared benchmark of speed, memory and token use gives developers numbers instead of marketing to choose between them.
Agent development environments, or ADEs, are desktop apps that wrap coding agents in a graphical interface. ADEMark is a new benchmark posted by the account zqkrq that tests them on the same machine and tasks. The first results were published on November 12.
Deep dive
The account zqkrq, display name zakra, posted on November 12 that ADEMark v1 is out. The test used the same Mac mini, the same tasks and the same model for every app, with a bare command line interface as the baseline.
The author wrote that they did not expect most of the apps to use fewer tokens than the command line on its own, or one app to sit at 16% CPU while doing nothing. They said this is the first of many runs and that updates will come more often.
The results table in the post is titled 15 ADEs, 12 tests, one Mac. It lists apps including super.engineering, Zeron, Paseo, Diri, T3 Code, Emdash, MonoCode, Synara, bb, Orca and Traycer, with Claude desktop, Capy, Cloudroom and Superset measured only in part. Tests cover speed, such as cold start and typing lag, weight, such as memory at rest and idle CPU, and tokens used. The table notes that each test is ranked on its own and that it is not a single score.
According to the table, the runs used Claude Opus 5.5 at high effort on a Mac mini M4 with 16 GB of memory. The post passed 160,000 views.
What to expect next
Watch for updated ADEMark runs with more apps and models, and responses from the developers of the tested environments. Independent reproduction on other hardware would make the results more useful. If the benchmark gains traction, app makers may start optimizing for its idle CPU and token measurements, and later versions may add a single overall score. Results with other models besides Claude Opus 5.5 would also show whether the rankings hold.