The Large Model Systems Organization develops large models and systems that are open, accessible, and scalable.
Latest Blog
See all posts
HiCache Across Multi-Parallelism and Distributed Serving
Agentic workloads repeatedly send long, mostly unchanged histories to the model. Radix cache reuses prefix KV in GPU memory, and HiCache extends that reuse to host memory and external storage.
Training SWE agents with Miles on MI355X: taming long-tail trajectories and complex sandbox coordination
We trained Qwen3-30B-A3B-Instruct with fully asynchronous agentic RL on AMD Instinct MI355X using Miles, SGLang, Megatron-LM, and Harbor, nearly quadrupling its pass@1 score on a 100-task stratified subset of SWE-bench Verified, from 7.3% to 27.5%.
SGLang security update: improving request isolation
On September 9, 2026, Cloudflare notified us of cross-request KV cache exposure in its GLM-5.3 deployment on Workers AI. We worked with Cloudflare to address the request-rejection path involved and strengthen related cache-lifetime handling in disaggregated serving.
Projects
View all projects
Our Sponsors & Partners
Backed by leading companies and institutions advancing AI research.
Voltage Park, NVIDIA, Nebius, Google Cloud, AtlasCloud, a16z, AMD, InnoMatrix, Laude Institute, Hyperbolic, NovitaAI, Verda Cloud, Sky9, Kaggle, MBZUAI, Together, RunPod, Anyscale, HuggingFace