A config mismatch leaked Cloudflare users' AI prompts to each other

Between August 27 and September 9, some Workers AI customers using the GLM-5.3 model received replies containing fragments of other people's requests. Cloudflare's writeup, published October 26, traces it to two faults that lined up. GLM-5.3 ran with a one-million-token context window, but the KV cache pool on its prefill GPUs held only about 681,000 tokens. Cloudflare knew the numbers did not match, but pre-launch tests returned normal HTTP 200 responses, so the team assumed the serving engine would reject oversized prompts cleanly.

It did not. The deeper fault was in SGLang, the inference engine, running in disaggregated mode with prefill and decode on separate GPU pools. When a prompt overflowed the pool, SGLang marked the request rejected and swapped in a one-token placeholder, but its admission queue checked only whether a request had finished, not whether it had been rejected. So the rejected request slipped through: prefill wrote a single KV page, decode had reserved eight pages and read across all of them, and seven still held leftover data from earlier requests. KV pages are not zeroed between uses, a normal performance choice, so the model returned a fluent answer stitched partly from someone else's prompt, with an HTTP 200 instead of an error. In total 155 responses were affected, each from a prompt longer than 681,280 tokens. A Knostic researcher reported it through HackerOne, and Cloudflare reproduced the leak in 25 minutes, pulled the model, and shipped a corrected configuration the same day. The SGLang bug itself had been filed back on July 6.

The longer-term work is the interesting part. Cloudflare prototyped per-page ownership tags that flag any read of a page a request does not own, at under 1% overhead on throughput and tail latency, and plans to formally verify its RDMA-based cache transfer protocol.

Why it matters

If you serve models on shared GPUs, this is the multi-tenancy failure worth fearing: not a clever attack but a quiet config mismatch plus an engine that answers 200 to a rejected request. Anyone running SGLang in disaggregated mode should move to v0.5.22 or later and check that the KV pool is at least as large as the context window.