The Large Model Systems Organization develops large models and systems that are open, accessible, and scalable.
Latest Blog
See all posts
SGLang security update: improving request isolation
On September 9, 2026, Cloudflare notified us of cross-request KV cache exposure in its GLM-5.3 deployment on Workers AI. We worked with Cloudflare to address the request-rejection path involved and strengthen related cache-lifetime handling in disaggregated serving.
miniTE Technical Report: Stable and Efficient Reinforcement Learning with Native NVFP4
Native NVFP4 reinforcement learning must remain stable under quantization and execution error while delivering practical efficiency gains. We investigate late-training instability by examining quantization formats, activation scaling, KV-cache precision, and framework differences that affect policy probabilities.
More Than DeepInfer: What SGLang Delivers
DeepSeek open-sourced DeepInfer, the inference engine behind its own service, together with the kernels it runs on. It sets the bar for how fast DeepSeek-V4.1-Flash can run, so we measured SGLang against it with DeepInfer's own benchmarks on the same hardware.
Projects
View all projects
Our Sponsors & Partners
Backed by leading companies and institutions advancing AI research.
Voltage Park, NVIDIA, Nebius, Google Cloud, AtlasCloud, a16z, AMD, InnoMatrix, Laude Institute, Hyperbolic, NovitaAI, Verda Cloud, Sky9, Kaggle, MBZUAI, Together, RunPod, Anyscale, HuggingFace