One transformer layer can match full RL fine-tuning

When you fine-tune a language model with reinforcement learning to make it better at math or code, the standard approach updates every parameter. A new paper led by Zijian Zhang asks whether that is necessary, and the answer is surprising: training a single transformer layer, chosen well, recovers most of the improvement that full-parameter RL delivers. The authors define a metric they call layer contribution, the fraction of the full RL gain you get by training just one layer while freezing the rest, and they measure it across seven models from the Qwen3 and Qwen2.5 families, three RL algorithms including GRPO and Dr. GRPO, and tasks that span math reasoning, code generation, and agentic decision-making.

The pattern holds across all of that. The layers that matter sit in the middle of the stack, while layers near the input and the output contribute much less, and the ranking of which layers help stays stable across different datasets, tasks, model families, and algorithms. That stability is the interesting part. It suggests RL post-training does not spread its effect evenly through the network but concentrates it in a region you can predict in advance.

Why it matters

If RL adaptation really lives in a handful of middle layers, you can train and store much smaller updates, which cuts the compute and memory cost of post-training. Anyone running RLHF or reasoning fine-tunes should watch whether this holds at larger scale, because it points to a cheaper recipe than updating the whole model.