Liquid AI's small encoders beat ModernBERT and run faster on CPU

Liquid AI has released two open encoder models, at 230M and 350M parameters, aimed at the workhorse jobs that do not need a generative model: classification, intent routing, PII detection, text scoring, and multilingual search. What is unusual is how they were made. Rather than train encoders from scratch, Liquid AI converted its LFM2.5 decoder models by swapping the causal attention mask for a bidirectional one, adding non-causal short convolutions with symmetric padding, and training with masked language modeling at 30% masking. A two-stage schedule takes the context window out to 8,192 tokens.

The results hold up against larger models. Across 17 tasks and 14 models spanning GLUE, SuperGLUE, and multilingual classification, the 350M encoder ranked fourth, behind only larger models including one at 3.5B parameters. The smaller 230M beat ModernBERT-base and every EuroBERT model despite its size. Speed is the other selling point: at 8,192 tokens on CPU, the 230M finished a forward pass in about 28 seconds against more than a minute and a half for ModernBERT-base, roughly 3.7 times faster, and it led at every sequence length tested. On GPU it pulls ahead from about 2,000 tokens onward. The write-up is here.

Why it matters

If you run retrieval or classification on CPU or at the edge, these give you better accuracy than ModernBERT at a smaller size and a clear speed advantage on long inputs, which is exactly where encoder cost usually hurts.