DPO cuts OCR degeneration by 59% on average with no human labels

A new Hugging Face post from Dharma AI argues that Direct Preference Optimization, the workhorse of post-training for chat models, is just as useful for structured generation tasks like OCR, and you do not need human preference labels to apply it. The trick is to use the model's own degenerate outputs, the repetitive token loops that supervised fine-tuning never quite eliminates, as the rejected side of a preference pair.

The numbers are striking. Across five OCR model families they tested, DPO reduced text degeneration rates by an average of 59.4% relative to supervised fine-tuning alone, with a peak of 87.6% on Nanonets-OCR2-3B, which dropped from 1.61% to 0.20%. Baseline degeneration rates across the five families varied wildly, from 0.60% to 33.96%, yet every family improved, in a range from 37.3% to 87.6%. The authors note that SFT optimizes for correct outputs but does not explicitly penalize degeneration, which is a categorical failure rather than a quality gradient. DPO with self-generated rejects gives the model a gradient to push against.

They are explicit about when this transfers. You need identifiable failure modes, reliable automated scoring, and enough inference data to mine for rejects. That covers a lot of structured-generation work, not just OCR.

Why it matters

If you fine-tune small models for structured tasks (extraction, captioning, parsing) and have given up on the residual degeneration that SFT leaves behind, this is a recipe you can run without paying for preference annotation. The pipeline only needs your own failure logs and an automatic checker.