The Large Model Systems Organization develops large models and systems that are open, accessible, and scalable.
Latest Blog
See all posts
DSpark in SGLang: Speculative Decoding with Confidence-Driven, Variable-Length Verification
Speculative decoding trades extra compute for fewer decode steps, and the trade sours as load grows: at batch size B with K speculative tokens the target verifies B K tokens every step, and past a po...
Agent-Assisted SGLang Development: An Initial Exploration
SGLang development increasingly goes beyond isolated code changes. The same repository now spans LLM serving, distributed runtime, GPU kernels, diffusion pipelines, model-specific execution paths, and...
Improving DeepEP MoE Load Balance in SGLang with Waterfill and LPLB
Mixture-of-Experts (MoE) models rely on Expert Parallelism (EP) to scale inference across multiple GPUs. In SGLang, DeepEP and EPLB provide high-performance serving under EP, but the workload seen by ...
Projects
View all projects
Our Sponsors & Partners
Backed by leading companies and institutions advancing AI research.
Voltage Park, NVIDIA, Nebius, Google Cloud, AtlasCloud, a16z, AMD, InnoMatrix, Laude Institute, Hyperbolic, NovitaAI, Verda Cloud, Sky9, Kaggle, MBZUAI, Together, RunPod, Anyscale, HuggingFace