sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [HiCache][PD] Allow DSA index-K elision under HiCache and disaggregation
- [Perf] Enable GDN ReplaySSM fold-every-commit for DFlash / DSpark verify
- [Fix] Keep one malformed request from killing a DP server
- [Fix] Keep a PD rebootstrapped request's cached_tokens from its first handoff
- xpu: build extend metadata for DLLM_EXTEND (fix dLLM decode crash on Intel XPU)
- [LoRA] Honor use_rslora from PEFT adapter config
- [Tokenizer] Optionally omit consumed processor input_ids from tokenized multimodal requests
- [6/N] [HiSparse] Keep draft DSA KV resident in logical token space
- [7/N] [HiSparse] Restore speculative verification reads and page release
- [8/N] [HiSparse][PD] Make large HiSparse host KV pools registrable for RDMA
- Docs
- Python not yet supported