A 350M model learns reliable JSON with 100 steps of reinforcement learning

A new Hugging Face tutorial shows how to make a small model produce schema-valid JSON and YAML using reinforcement learning, without reaching for a larger model or a longer prompt. The target is LFM2.5-350M, a 350-million-parameter model, and the method is Group Relative Policy Optimization (GRPO) through the TRL library. Instead of one accuracy label, training uses three reward functions: one for whether the output parses as JSON, one for the expected number of fields, and one for passing full JSON Schema validation, weighted so schema validity counts most.

The numbers are modest but honest. On the IFStruct benchmark the model went from 22.6% to 29.7%, and JSON-only compliance rose from 18.0% to 31.9%. The run took about 100 steps on roughly 500 samples and fits on a free-tier GPU, training only about 6 million parameters, 1.66% of the model, through LoRA. The post ships the full pipeline and code.

Why it matters

If you build agents or data pipelines that depend on structured output, a small local model that reliably returns valid JSON can replace a larger API call for that step. This shows the fix can be a short, cheap RL run rather than a bigger model, and gives you a recipe to try on your own schema.