An LLM edits a model's weights by hand to fix its reasoning

Most ways of changing a neural network go through training: you run data forward, compute a loss, and let gradients nudge the weights. A new write-up from researchers at MIT CSAIL, Sparks of Weight-Space Programming, tries something stranger. They give a frontier model, GPT-6 Astra, the raw numbers of a small network and ask it to reason about what each part computes, then edit the weights by hand to match a plain-language spec. No optimizer, and in most experiments no gradients or training loop at all.

The headline result is a repair job on Qwen3.5-0.8B, a small model known to get stuck in reasoning loops. Astra traced the loop to a single direction written by the MLPs of layers 15 and 19, then subtracted that direction from 64 output columns in each layer. That touched 131,072 weights, about 0.016% of the model. With 50 forward passes to check its work, the edit raised the model's MATH-500 score from 7.9% to 48.1%, GSM8K from 13.8% to 67.1%, and instruction-following on IFEval from 21.0% to 62.3%. The share of answers that ran into the 32,768-token limit dropped from 93% to 33%. In a separate control task, changing 3.5% of a policy network's weights made a simulated car clear a ramp in all 1,024 test episodes, where the original never succeeded once in 256.

The authors are careful about what this is not. They call these very early results, limited by how much of a network fits in the model's context. When they tried the same trick on Gemma 3 270M, which rarely loops, they could not lift its math score at all. Their conclusion is worth keeping: editing weights could remove an obstacle that hid an ability, but it could not add an ability the model lacked.

Why it matters

If models can read and repair weights directly, interpretability stops being only a diagnostic and starts becoming a way to fix behavior without retraining. For anyone working on alignment or on shipping small models, that is a tool to watch, but the Gemma result is the guardrail: do not expect weight edits to conjure capabilities that were never there.