sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Perf] [NPU] dsa: Token shard composes with the DCP KV gather
- Lazily initialize NCCL2 process groups
- [Fix] models: Fix Phi and Persimmon startup crash under breakable prefill CUDA graph
- [Fix] [NPU] diffusion: Clean up warnings
- [Fix] tokenizer_manager: Reject prompts that tokenize to zero tokens
- [Fix] logprob: preserve the maximum valid input token ID
- [Fix] io_struct: Reject per-request lists whose length differs from the batch size
- [Fix] io-struct: validate batch bootstrap routing columns
- [Fix] io-struct: reject empty embedding token IDs
- [Fix] tokenizer_manager: Reject generation requests on an embedding server
- Docs
- Python not yet supported