The Large Model Systems Organization develops large models and systems that are open, accessible, and scalable.
Latest Blog
See all posts
LMSYS Org Launches SGLang Consortium with NVIDIA, AMD, Intel, Google, Meta, Cloudflare, Modal and RadixArk to Advance Open-Source AI Inference
Hardware, model and infrastructure leaders join SGLang’s maintainers to advance efficient inference for growing AI ecosystem
SGLang-Neuron v0.1.0: SGLang's Serving Stack on AWS Trainium
Today we are releasing SGLang-Neuron v0.1.0, the first public release of SGLang's backend for AWS Trainium. Trainium is a portfolio of AWS AI chips that delivers the best economics for AI training and inference at scale. It follows the closed beta the SGLang team announced in October.
Muse Spark Open Ultrafast: Pushing beyond 1,000 Tokens per second
In this post, we explain why SGLang (and ML infra in general) is moving towards larger GPU kernels, called megakernels, and what makes one different from a normal kernel. We built one for Muse Spark Open, from scratch in CUDA C++ and PTX, and it takes decoding 517 → 1,109 tok/s on Blackwell GPUs.
Projects
View all projects
Our Sponsors & Partners
Backed by leading companies and institutions advancing AI research.
Voltage Park, NVIDIA, Nebius, Google Cloud, AtlasCloud, a16z, AMD, InnoMatrix, Laude Institute, Hyperbolic, NovitaAI, Verda Cloud, Sky9, Kaggle, MBZUAI, Together, RunPod, Anyscale, HuggingFace