A KV cache leak in Cloudflare’s Workers AI deployment of GLM-5.3 sent fragments of other customers’ requests back to 155 responses between August 27 and September 9, 2026. The responses came back as ordinary HTTP 200s with coherent text, and the cause was a configuration choice Cloudflare knew about before launch, colliding with a bug in the open-source SGLang inference engine.
Cloudflare and the SGLang team published coordinated disclosures on October 26. Both posts are detailed, and the mechanism is worth understanding even if you have never touched Workers AI, because the same engine serves open models at a lot of providers and inside a lot of companies.
I read Cloudflare’s incident write-up, the SGLang team’s security update and the SGLang release notes. Everything below comes from those, and I have flagged where the posts leave questions open.
What Happened: The Cloudflare GLM-5.3 Leak in Numbers
Cloudflare launched @cf/zai-org/glm-5.3 on Workers AI on August 27, 2026, with a 1,048,576-token context window but a prefill KV cache pool that held only 681,280 tokens. Any prompt between those two numbers fit the advertised window but did not fit in memory.
On September 8 at 23:11, security researcher Gadi Evron of Knostic reported through Cloudflare’s HackerOne program that long prompts sometimes returned other users’ content, with a reproduction script attached. The response from there was fast.
| Time (Sep 9, 2026) | Event |
|---|---|
| Morning | Cloudflare begins verifying the HackerOne report |
| 13:22 | Security incident declared; leak reproduced from two independent accounts within 25 minutes |
| 14:00 | Model set to private, requests return HTTP 400 (within 40 minutes of the incident) |
| 19:40 | Model restored after config fix and patched engine are validated |
(Source: Cloudflare blog, October 26, 2026)
The impact figures from Cloudflare:
- 155 affected responses, all for prompts longer than 681,280 tokens.
- A “small number” of customers received responses containing other customers’ data.
- No other Workers AI model was affected.
- “We found no evidence of deliberate exploitation”, and the remaining requests, after excluding Cloudflare’s and the researcher’s own testing, were “consistent in timing and volume with ordinary long-context use.”
That last point cuts both ways. Nobody was attacking the service, which is good news. It also means real customers sending real long-context prompts got other people’s data back, which is the worst kind of bug for a shared inference service.
Thirteen days of exposure, 155 responses, and a 40-minute takedown once someone reported it: the response was good, the detection was not.
How a KV Cache Leak Happens in SGLang
To follow the bug you need two concepts.
The KV cache stores the key and value tensors a transformer computes for every token it has already processed, so the model does not recompute them for each new token. It lives in GPU memory, carved into fixed-size pages.
Disaggregated prefill/decode (PD) serving splits inference across two pools of GPUs. Prefill workers process your prompt and build the KV cache. Decode workers receive that cache over RDMA and generate output tokens. It is how providers squeeze more throughput out of long prompts, and SGLang supports it with transfer backends like Mooncake, NIXL and MORI.
Here is the chain Cloudflare describes, step by step:
- A prompt longer than 681,280 tokens arrives. It fits the context window, but not the prefill KV pool.
- SGLang’s scheduler (
Req.set_finish_with_abort) marks the request as rejected and swaps the prompt for a one-token placeholder instead of returning an error. - The disaggregation admission queues checked whether a request was finished, not whether it was rejected. The placeholder passed.
- Prefill sent one KV page to decode. Decode had reserved eight pages.
- Seven of those eight pages were never overwritten for this request. They still held earlier requests’ data, because KV pages are not zeroed between requests.
- Decode generated a full response by attending over those stale pages, and returned HTTP 200.
The part that bothers me most is step 6. The failure mode was not a crash, a timeout or garbage output. It was fluent text built from someone else’s prompt, with a success status code.
The upstream bug was not new. Cloudflare says it was reported to SGLang as issue #30233 on July 6, 2026, and that Cloudflare had flagged similar issues since v0.5.10. On a single-node SGLang deployment, the same rejection path returned an error. Only the disaggregated path leaked.
The leak needed both halves: a pool smaller than the context window, and an admission check that let a rejected request through to decode.
The Second SGLang KV Cache Bug: Late RDMA Writes
The SGLang team’s post describes a separate, quieter race in the same PD path, and it affects far more deployments.
When a request aborts mid-transfer, decode could free its KV slots while RDMA writes from prefill were still in flight. The allocator could hand those slots to a new request, and then the late writes would land on top of it. In the post’s words, “launched RDMA work cannot be cancelled, for push and pull alike.”
The scope is wide: PD deployments from the v0.4.x line in early 2025 through v0.5.21, or v0.5.22 for the MORI backend. Longer contexts widen the window.
The SGLang team is careful about the threat model. They describe it as one-way leakage. An attacker who aborts requests can raise the odds, but cannot choose the victim, cannot choose where the writes land, and learns nothing from the victim’s output. They also say the bug is silent, so an operator may not be able to tell after the fact whether exposure happened.
The fix is a deferred release: decode holds an aborted request’s slots until every prefill rank acknowledges the abort, or a timeout fires. The SGLang team calls that timeout “a practical backstop, not a proof”, which is the most honest sentence in either post.
They also measured the cost. Across 21 runs on 2×H100, throughput differences stayed within ±2%, and a separate H200 sweep measured −0.38%, not statistically significant.
If you run SGLang with prefill/decode disaggregation and have not upgraded since early October, assume you are exposed to this race.
SGLang Fix Checklist for Operators
Cloudflare’s advice is blunt: “We highly recommend all operators update SGLang dependencies to v0.5.22 or later.” The SGLang post adds backend-specific detail. Combined, this is what to check.
| Check | What to do | Why |
|---|---|---|
| SGLang version | Upgrade to v0.5.22+ (Oct 19, 2026); v0.5.23 for the NIXL/MORI failed-send fix | Earlier branches were not backported |
| KV pool vs context length | Read pool size from startup logs on every prefill and decode node | The mismatch that started the Cloudflare leak |
| Deferred KV release | On by default since v0.5.21; set SGLANG_DISAGGREGATION_DEFERRED_DECODE_KV_RELEASE=1 on v0.5.19–v0.5.20 | Stops late RDMA writes landing on reused slots |
| Corruption detection | Enable --disaggregation-enable-kv-checksum on both prefill and decode | Adler-32 check of transferred KV, off by default |
| Router aborts | Use a router that cancels both halves of a PD pair, such as sgl-router PD mode | Unpaired aborts create the race window |
(Sources: Cloudflare blog, SGLang security update, SGLang releases, October 2026)
SGLang also recommends watching the sglang:decode_deferred_kv_release_total{outcome} metric. A healthy deployment reports mostly drained; a high timeout ratio means every abort waits out the full hold.
One detail to watch: the SGLang release notes for v0.5.20, v0.5.21 and v0.5.22 carry no security notes at all. If your upgrade process relies on reading release notes for “security”, you would have missed this entirely.
Upgrade first, then compare KV pool size to context length on every node. Those two checks cover the Cloudflare failure and the wider race.
Why Cloudflare’s Tests Missed the KV Cache Leak
This is the lesson I would put in front of any team running inference, including teams that only self-host a model for internal coding agents.
Cloudflare says plainly: “We were aware of this mismatch before launch.” It did not treat it as a defect because SGLang sizes the pool from leftover GPU memory at startup and nothing forces it to match the context length. And the pre-launch tests “only validated HTTP response codes and engine performance at long contexts.”
Every test passed because every test asked the wrong question. A status-code check cannot see a correct-looking answer to the wrong prompt.
Cloudflare’s own remediation shows what the right test looks like. It validated the fixes with canary markers: put a unique string in one request, then check that it never appears in another. That is cheap to build and catches exactly this class of bug. Cloudflare has since made every disaggregated deployment change pass checks on KV pool size versus context length, deferred release on both roles and transfer batch bounds.
Longer term, Cloudflare prototyped per-page integrity tags checked at read time, which cost under 1% in throughput and tail latency, but disabled it in mid-August for reliability reasons. It is now building a production version with the SGLang team, and plans formal verification of its RDMA-based KV transfer protocol.
If your inference tests do not check that output belongs to the input, you are testing availability, not isolation.
What This KV Cache Leak Means If You Use Hosted Open Models
Most developers will never run SGLang. You call an API, and the provider runs the engine. So what do you do with this?
First, know who is in scope. If you called @cf/zai-org/glm-5.3 on Workers AI between August 27 and September 9 with prompts over roughly 681,000 tokens, you were in the affected group. GLM-5.3 on Workers AI is billed at $1.40 per million input tokens, $0.26 cached and $4.40 output, with a 1,048,576-token window, according to Cloudflare’s model page, so million-token prompts were not an exotic use case. The Cloudflare post does not say whether affected customers were contacted. If you think you were affected, ask Cloudflare directly.
Second, think about what goes into a 700,000-token prompt. In 2026 that is usually a coding agent stuffing an entire repository, its config and its logs into context. That is where .env files, internal URLs and customer data end up. This incident maps directly onto OWASP’s LLM02:2025 Sensitive Information Disclosure, whose first example scenario is a user receiving a response containing another user’s data.
Third, the provider-side controls you can apply are the same ones covered in the AI coding agent security guide: keep secrets out of agent context, rotate anything that may have been sent, and scope API keys tightly. The OpenCode Console breach earlier this month was a different failure, but it taught the same lesson about what happens when keys and prompts live on someone else’s infrastructure.
I do not think this is a reason to stop using hosted open models. Cloudflare’s write-up is one of the most detailed incident reports I have read from an inference provider, and the takedown took 40 minutes. A provider that publishes the scheduler function name and the PR numbers is one I trust more after an incident, not less.
It is a reason to stop assuming that million-token context windows are free. They push inference engines into memory-pressure paths that see far less traffic and testing than a 10,000-token chat.
The Bigger Picture for AI Infrastructure
Two things stand out to me.
The first is that a single open-source engine now sits under a lot of AI products. When SGLang’s disaggregated path had a race that dates back to early 2025, every provider using that path inherited it, and only one has published an incident so far. The SGLang post says the researcher’s own report will follow, which may say more.
The second is that the bug class is new for most application developers. We are used to thinking about SQL injection and broken access control. A tenant isolation failure inside GPU memory, triggered by prompt length, is not on most threat models yet.
As the routers and agents we use get more automated, as with the GitHub Copilot Fusion model routing covered in the companion post, more of the stack sits out of view. Cloudflare’s disclosure is a useful look at what can go wrong underneath, and at what a good response looks like when it does.
