MIT CSAIL Researchers Say GPT 6 Astra Fixed Qwen3.5 0.8B's Reasoning Loops by Directly Editing 0.016% of Its Weights, With No Training
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 馃У Blog:
Context
If frontier models can read and rewrite another network's weights directly, it opens a new way to repair, tune and inspect models without any training run.
Models are normally improved by training, which adjusts weights using gradients computed from data. Qwen3.5 0.8B is a small open weight model from Alibaba's Qwen team whose model card warns about reasoning loops. MATH 500 and IFEval are standard benchmarks for math problem solving and instruction following.
Deep dive
Rikiya Takehi (@rikiyatakehi) posted on Nov 2 that his team asked GPT 6 Astra to edit the weights of Qwen3.5 0.8B to improve its math. According to the post, Astra directly changed only 0.016% of the weights, fixing the small model's reasoning loops and improving math up to 6x and instruction following 3x, without gradients or training. The team calls the idea Weight Space Programming.
The attached blog screenshot lists the authors as Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab of MIT CSAIL. Its summary gives more detail: with the same sampling settings, the weight edit raises MATH 500 from 7.9% to 48.1%, which is where the roughly 6x figure comes from, and IFEval by 3x. It also says that just by looking at raw weights, Astra can tell what a network was trained to do, and that it can write small game playing networks from scratch that outplay networks trained with PPO for 100 million steps.
These are the team's own results and had not been independently reproduced at the time of posting. A small model with a known failure mode such as looping is also an easier target than a large, well tuned model, so the size of the gain may not carry over.
Still, the claim is unusual. Weights are usually treated as opaque numbers that only training can change well. If a language model can find the specific weights behind a bug and rewrite them, that would be useful both for fixing models cheaply and for interpretability research that tries to explain what weights do.
What to expect next
Watch for the full paper and code, and for other groups trying the same edits on larger models. Key questions are whether the gains hold on other benchmarks, whether the edits cause hidden side effects, and whether models other than GPT 6 Astra can do it. If the approach generalizes, expect interpretability and model repair work to start using frontier models as weight editors.