← Tuesday, November 17 deck

31Open source1 source posting

Clement Delangue: An Open Weights Qwen3.8 27B Adapted for the pi Coding Agent More Than Doubles Its SWE Bench Pro Hard Score

Introducing Qwen3.8-27B-pi-adapted 🤗 We adapted Qwen's model specifically for the pi coding agent using reinforcement learning and a LoRA adapter. The results, measured as pass@1: 🚀 SWE-bench Pro (Hard 51): 29.4% → 62.7% 💻 Terminal-Bench 2.1: 39.5% → 55.8% ⚡ 55–62% fewer tool calls per task This is the power of open models: you can adapt them to YOUR tools and YOUR use cases. Let’s build! 💛 Open weights: t.co/aH4cl2qnED Methodology:

Bar chart comparing Qwen3.8 27B in gray with the pi adapted version in yellow: 29% to 63% on SWE Bench Pro Hard 51 subset, up 34 points, and 40% to 56% on Terminal Bench 2.1, up 16 points, next to a panel reading 55 to 62% fewer tool calls.

Context

Why it matters

A small LoRA adapter trained with reinforcement learning inside one agent tool more than doubled a mid size open model's score on hard coding tasks, which shows people can fit open models to their own tools cheaply.

Hugging Face cofounder Clement Delangue announced the model on Nov 17 with links to open weights and a methodology post. The methodology describes harness adapters: one small LoRA adapter per model and agent harness, trained with reinforcement learning on verifiable tasks inside unmodified agents such as pi, opencode and hermes. pi is an open source coding agent.

Deep dive

Clement Delangue posted that his team adapted Qwen's 27B model specifically for the pi coding agent, using reinforcement learning and a LoRA adapter. He reported pass@1 results: on a hard 51 task subset of SWE Bench Pro the score rose from 29.4% to 62.7%, and on Terminal Bench 2.1 it rose from 39.5% to 55.8%. He also said the adapted model makes 55% to 62% fewer tool calls per task. The weights are open and linked from the post.

The methodology link points to a Hugging Face blog post published Nov 6 on harness adapters. It explains that the same model behaves differently in different agent harnesses, so the team trains one small LoRA adapter per harness on top of a base model. Training tasks include repairing bugs, answering factual questions, building projects from scratch and extending existing code, many generated from open coding sessions shared on the Hub. A proxy called safeagents checks every tool call during training and blocks attempts to cheat, such as reading the hidden tests, which then score zero.

These are results reported in the post on the team's own evaluation setup, including a subset of SWE Bench Pro rather than the full benchmark. They had not been independently reproduced in the posts reviewed here. The gain also applies to use inside pi; replies asked whether it carries over to other agents.

What to expect next

Watch for independent tests of the adapter on the full SWE Bench Pro and in everyday use inside pi, and for adapters for other harnesses such as opencode and hermes, which the blog says are part of the same program. Replies asked for versions built on other Qwen models. If the approach holds up, more teams may publish adapters tuned to their own agents and tools rather than training whole models.