Gemma 4 12B drops the vision encoder for a single matrix multiplication

Google has added a 12B model to the Gemma 4 family, and the interesting part is what is missing. The post says vision and audio inputs flow into the language model through a "lightweight embedding module consisting of a single matrix multiplication, positional embedding and normalizations," with raw audio projected straight into the same dimensional space as text tokens. There is no separate vision encoder like SigLIP, and no audio frontend like Whisper, which is the usual shape for open multimodal models. The Google announcement calls the architecture "encoder-free."

The 12B is Apache 2.0, downloadable from Hugging Face and Kaggle, and supported in Transformers, llama.cpp, MLX, SGLang, and vLLM out of the box. Google says it lands close to its larger 26B mixture-of-experts model on standard benchmarks while running on a consumer laptop with 16GB of unified memory, and Multi-Token Prediction drafters are bundled in to reduce decoding latency. The post does not publish exact benchmark numbers per task.

This sits alongside the original April Gemma 4 sizes (an effective 2B, an effective 4B, the 26B MoE, and a 31B dense model), and Google notes the family has crossed 150 million downloads. The encoder-free idea is the bet that matters longer term, because it folds more of the multimodal pipeline into one optimizable graph.

Why it matters

If you build local AI features into apps, this 12B is the first Gemma you can ship with full multimodal in a 16GB unified-memory budget, and the encoder-free design simplifies what you actually integrate. Verify the quality claims on your own data before swapping out a SigLIP-backed pipeline.