The Large Model Systems Organization develops large models and systems that are open, accessible, and scalable.
Latest Blog
See all posts
Muse Spark Open Ultrafast: Pushing beyond 1,000 Tokens per second
In this post, we explain why SGLang (and ML infra in general) is moving towards larger GPU kernels, called megakernels, and what makes one different from a normal kernel. We built one for Muse Spark Open, from scratch in CUDA C++ and PTX, and it takes decoding 517 → 1,109 tok/s on Blackwell GPUs.
Unified Memory in SGLang: One Arena for Every Token Lifetime
Attention layers need their past to compute the next token, but not every layer remembers it the same way:
XM-Sparse: Pushing the Throughput Frontier of Sparse Attention with Netpreme X-Mem™
XM-Sparse integrates Netpreme X-Mem™ with SGLang HiSparse. High-bandwidth access to offloaded KV caches reduces decode latency, while the added capacity supports larger batches and higher concurrency.
Projects
View all projects
Our Sponsors & Partners
Backed by leading companies and institutions advancing AI research.
Voltage Park, NVIDIA, Nebius, Google Cloud, AtlasCloud, a16z, AMD, InnoMatrix, Laude Institute, Hyperbolic, NovitaAI, Verda Cloud, Sky9, Kaggle, MBZUAI, Together, RunPod, Anyscale, HuggingFace