LMSYS Org

The Large Model Systems Organization develops large models and systems that are open, accessible, and scalable. It is currently run by students and faculty members from UC Berkeley Sky Lab.

Vicuna


A chatbot impressing GPT-4 with 90%* ChatGPT quality, available in 7B/13B/33B sizes.

Chatbot Arena


Scalable and gamified evaluation of LLMs via crowdsourcing and Elo rating systems.

SGLang


Efficient interface and runtime for complex LLM programs

LMSYS-Chat-1M


A large-scale real-world LLM conversation dataset.

FastChat


An open platform for training, serving, and evaluating LLM-based chatbots.

MT-Bench


A set of challenging, multi-turn, and open-ended questions for evaluating chatbots.

Arena Hard Auto


An automatic pipeline converting live data to high-quality benchmarks for evaluating chatbots.

RouteLLM


An open-source framework for serving and evaluating LLM routers.