LMSYSORG

The Large Model Systems Organization develops large models and systems that are open, accessible, and scalable.

15+Projects
75K+GitHub Stars
1000+Contributors

Latest Blog

See all posts
Blog

SGLang and Miles Add Day-0 Support for NVIDIA Nemotron 3 Ultra for Long-Running Autonomous Agents

We are excited to announce that SGLang and Miles support NVIDIA Nemotron 3 Ultra on Day 0\. Agentic AI systems are moving from short prompt-response interactions to persistent workflows that plan, us...

June 4, 2026
Blog

Heterogeneous CPU + GPU EPD Disaggregation to Boost VLM Serving

TL;DR We enabled heterogeneous Encode-Prefill-Decode (EPD) disaggregation via Dynamo and SGLang for Vision-Language Models (VLMs). By offloading vision encoding tasks to CPUs (the easiest-getting CPU...

May 29, 2026
Blog

Win on TCO: How AMD Instinct™ MI355X Achieves Cost-Competitive Distributed Inference Through SGLang with MoRI

The SGLang and AMD team has worked closely to unlock competitive Total Cost of Ownership (TCO) for large-scale DeepSeek-R1 disaggregated inference on AMD Instinct™ MI355X GPUs. Building on SGLang's se...

May 28, 2026

Projects

View all projects
SGLang
SGLangstars

A fast serving engine for LLMs and VLMs.

SGLang
Chatbot Arena
Chatbot Arena
Chatbot ArenaGraduated

A benchmark platform for LLMs that features anonymous, randomized battles in a crowdsourced manner.

FastChat
FastChatstars

An open and scalable platform for training, finetuning, serving, and evaluating LLM-based chatbots.

FastChat
Vicuna
Vicuna
Vicuna

An open-source chatbot impressing GPT-4 with 90%* ChatGPT quality.

RouteLLM
RouteLLMstars

A framework for serving and evaluating LLM routers.

RouteLLM

Our Sponsors & Partners

Backed by leading companies and institutions advancing AI research.

Voltage Park, NVIDIA, Nebius, Google Cloud, AtlasCloud, a16z, AMD, InnoMatrix, Laude Institute, Hyperbolic, NovitaAI, Verda Cloud, Sky9, Kaggle, MBZUAI, Together, RunPod, Anyscale, HuggingFace