The Large Model Systems Organization develops large models and systems that are open, accessible, and scalable.
Latest Blog
See all posts
SGLang security update: improving request isolation
On September 9, 2026, Cloudflare notified us of cross-request KV cache exposure in its GLM-5.3 deployment on Workers AI. We worked with Cloudflare to address the request-rejection path involved and strengthen related cache-lifetime handling in disaggregated serving.
More Than DeepInfer: What SGLang Delivers
DeepSeek open-sourced DeepInfer, the inference engine behind its own service, together with the kernels it runs on. It sets the bar for how fast DeepSeek-V4.1-Flash can run, so we measured SGLang against it with DeepInfer's own benchmarks on the same hardware.
Announcing the SGLang AWS Trainium Plugin Beta
In collaboration with AWS and alongside the TorchNeuron launch, we are announcing the beta of the SGLang Trainium plugin, enabling both inference serving and RL rollouts on AWS Trainium.
Projects
View all projects
Our Sponsors & Partners
Backed by leading companies and institutions advancing AI research.
Voltage Park, NVIDIA, Nebius, Google Cloud, AtlasCloud, a16z, AMD, InnoMatrix, Laude Institute, Hyperbolic, NovitaAI, Verda Cloud, Sky9, Kaggle, MBZUAI, Together, RunPod, Anyscale, HuggingFace