Rohan Paul says a new reinforcement learning post train of Qwen3.8 27B halves its hallucination rate
Some of the most interesting agent results this year are coming out of post-trains of small open models. A new RL post-train Qwen3.8-27B model trained with an open-source environments cuts the hallucination rate on AA-Omniscience from 30.4% to 15.1%. the compressed copy of the mla version fits on my single 24 GB RTX 4090 and holds about 3.5x more context than base Qwen. It beats GPT 6 Luna-medium on DeepSWE, placing in between Luna 6 medium and high. scores higher on Terminal-Bench 2.1 while writing a median 23.8K output tokens per solved task, where base Qwen writes 54.9K. Tried an unreleased version of this model, and really felt the comparative intelligent density, that it writes a lot less. tops Muse Glimmer 30B and OpenAI’s GPT-6 Luna (high) on Terminal-Bench, τ³-bench, ClawEval and BFCL v4 (Berkeley Function Calling Leaderboard), though Luna still wins DeepSWE at 53.1%. They also released an MLA version, which is the same model with its 16 full-attention layers converted to multi-head latent attention, which shrinks the memory each token takes from 64 KiB to 18 KiB so far more or longer sessions fit on one GPU.
Context
If confirmed, small open models tuned with open training environments could match much larger closed models on agent tasks.
This is a secondhand post from Rohan Paul, who writes an AI newsletter, and he does not name the team or link the release. He says he tried an unreleased version, so the figures are unverified and the model could not be identified for this deck.
Deep dive
On October 20, Rohan Paul posted that some of the most interesting agent results this year are coming from post trains of small open models. Post training means further training a released model, here with reinforcement learning, where the model is rewarded for completing tasks in practice environments.
He said a new reinforcement learning post train of Qwen3.8 27B, trained with open source environments, cuts the hallucination rate on AA Omniscience, a knowledge test from Artificial Analysis, from 30.4% to 15.1%. He said it beats GPT 6 Luna medium on DeepSWE, placing between Luna medium and high, and tops Muse Glimmer 30B and GPT 6 Luna high on Terminal Bench 2.1, tau3 bench, ClawEval and the Berkeley Function Calling Leaderboard, though Luna still wins DeepSWE at 53.1%. He said it writes a median 23.8 thousand output tokens per solved Terminal Bench task, against 54.9 thousand for base Qwen.
He also described a second release that converts the model's 16 full attention layers to multi head latent attention, cutting memory per token from 64 KiB to 18 KiB. He said a compressed copy fits on a single 24 GB RTX 4090 and holds about 3.5 times more context than base Qwen.
He did not name who built the model or link to it, and said he tried an unreleased version. None of the figures could be checked.
What to expect next
Watch for the team behind the model to publish it with its training environments and evaluation details, which would let others check the hallucination and benchmark figures. If the results hold, expect more groups to post train small open models such as Qwen3.8 27B for agent work. Independent tests on AA Omniscience and Terminal Bench would be the clearest confirmation.