The Large Model Systems Organization develops large models and systems that are open, accessible, and scalable.
Latest Blog
See all posts
More Than DeepInfer: What SGLang Delivers
DeepSeek open-sourced DeepInfer, the inference engine behind its own service, together with the kernels it runs on. It sets the bar for how fast DeepSeek-V4.1-Flash can run, so we measured SGLang against it with DeepInfer's own benchmarks on the same hardware.
Announcing the SGLang AWS Trainium Plugin Beta
In collaboration with AWS and alongside the TorchNeuron launch, we are announcing the beta of the SGLang Trainium plugin, enabling both inference serving and RL rollouts on AWS Trainium.
Elastic Expert Parallelism Scaling in SGLang: Growing a Live MoE Deployment with NIXL
Inference deployments operate under changing load and must adapt their capacity while maintaining throughput and latency targets. For large Mixture-of-Experts (MoE) models, the standard options are horizontal scaling, which adds another complete replica, and vertical scaling, which restarts the deployment with a wider expert-parallel topology.
Projects
View all projects
Our Sponsors & Partners
Backed by leading companies and institutions advancing AI research.
Voltage Park, NVIDIA, Nebius, Google Cloud, AtlasCloud, a16z, AMD, InnoMatrix, Laude Institute, Hyperbolic, NovitaAI, Verda Cloud, Sky9, Kaggle, MBZUAI, Together, RunPod, Anyscale, HuggingFace