The Large Model Systems Organization develops large models and systems that are open, accessible, and scalable.
Latest Blog
See all posts
Serving GLM5.2 NVFP4 Agentic Workload with SGLang: Reaching 500 TPS in 2 Weeks
- More than 500 TPS on 8xB300 (bs=1) - Sync free speculative decoding for GLM 5.2 MTP - Built-in IndexShare MTP with Spec V2 - 2.33x faster TopK-V2 for ISL 80k - Indexer prologue fusion - Gemm kernels...
Bringing DeepSeek-V4 Flash RL Training to AMD Instinct MI355X GPUs with Miles
DeepSeek-V4 RL is now supported in Miles on AMD Instinct™ MI355X GPUs with ROCm™! RL requires SGLang rollout and Megatron training to implement the same policy closely enough that token probabilities ...
Accelerating SGLang HiCache with Netpreme X-Mem™ MPU
Netpreme X-Mem™ Memory Processing Unit (MPU) makes SGLang HiCache faster and more scalable by augmenting the slower Host DRAM offload tier with a purpose-built high-bandwidth KV memory tier. - Pr...
Projects
View all projects
Our Sponsors & Partners
Backed by leading companies and institutions advancing AI research.
Voltage Park, NVIDIA, Nebius, Google Cloud, AtlasCloud, a16z, AMD, InnoMatrix, Laude Institute, Hyperbolic, NovitaAI, Verda Cloud, Sky9, Kaggle, MBZUAI, Together, RunPod, Anyscale, HuggingFace