sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- fix(hicache): fail-soft SWA prefetch degrade with Mamba guard, prefetch last_hash recompute
- [Bugfix] Handle empty DFlash2 proposals with a one-token verify window
- [Unified Cache] Handle external-linker KV load failure instead of crashing the scheduler
- [Multimodal] Materialize split views before caching embeddings
- fix(mem_cache): eliminate chunked-prefill radix-insert race corrupting QSA KV pages (#38319)
- [RFC / Feature]: Zero-Copy In-Place KV-Cache Compactor via In-Situ Cycle Permutations (1.65x Speedup, 0 Bytes Aux VRAM)
- Support namespaced function tools in the Responses API
- [Bug] /v1/score returns validation error for single-label cross-encoder rerankers (e.g. bge-reranker-v2-m3)
- fix: wrap scalar cross-encoder scores into per-item rows for `/v1/score`
- [HiCache] Isolate Mooncake KV objects by kv_cache_dtype via tenant_id
- Docs
- Python not yet supported