sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] rust-server: Prepend BOS for text prompts when tokenizer relies on add_bos_token
- [Fix] mem_cache: Bound a chunked continuation past decode headroom by free rows
- [Perf] [NPU] sampling: threshold top-k/top-p renormalization instead of a full-vocabulary scatter
- [Perf] [NPU] sampling: fuse top-k/top-p truncation into a candidate-set pass
- [Fix] rust-server: Restore add_bos_token as the Python tokenizer does
- [Perf] cosmos3: batch clean refresh and enable VAE fast paths for Bimanual model
- [Fix] xing4_0: Number streamed tool calls by ordinal and keep Optional[str] args as strings
- [Fix] [NVIDIA] qwen3.5: Give text-only Qwen3.5 checkpoints the Qwen3 MoE family overrides
- [Feat] kv-cache: Metrics for write latency distribution from L2 to L3
- [Bug] FA3 SM90 FP8 two-level accumulation rebuild is ~2.6x slower for paged head_dim=256
- Docs
- Python not yet supported