Cross-request KV cache exposure: a lesson for companies deploying LLMs

A concrete case: a configuration mismatch

According to Cloudflare, a configuration mismatch in the GLM 5.3 model running on Workers AI triggered a bug in SGLang, the engine used to serve language models. The result was the exposure of KV cache pages across different requests. In practice, parts of the processing for one query could become accessible in the context of another. Cloudflare describes in its blog post how the issue was addressed.

Why KV cache is an isolation problem, not only a model problem

The KV cache stores intermediate results of token processing, so they do not have to be recalculated at every generation step. It is part of the infrastructure that serves the model, not part of its weights. This is why a review of model safety, behavior or content filters alone will not detect such a flaw. What is needed is control of the execution layer: how memory is shared, how sessions are isolated and which version of the inference engine is in use.

How to verify isolation in your own deployment

Companies deploying LLMs should go through several steps:

1. identify which inference engine and version serve the model, and whether the vendor has published fixes for that version,
2. check whether the model configuration in production matches the recommendations for that version,
3. test concurrent requests from different users or tenants and verify that responses do not contain content from other sessions,
4. record the test results as evidence of control.

A common pitfall is assuming that the cloud provider is responsible for everything. Responsibility for configuration and verification usually remains with the deploying organization.

Where the legal risk lies

If the requests contain personal data, leakage between sessions may constitute a security incident under the GDPR in the EU. In particular, Article 32 concerns the security of processing, and Article 33 requires notifying the supervisory authority within 72 hours when there is a risk to individuals. Pointing to an external component as the source of the error does not release the controller from this responsibility.

Summary

The case shows that the security of AI systems depends on the entire stack: configuration, the inference engine and data isolation between users. Companies should verify these elements rather than relying solely on the model provider's assurances. Cloudflare's source describes the technical details and the remediation.

Lub System helps B2B companies implement AI, automation and IT solutions end-to-end - from strategy to deployment. See our services or get in touch to discuss your case.