sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] [AMD] model-loader: register full allocations for remote weight loading
- [Perf] wan2.2-s2v: Build the per-request shapes once, as Python ints
- [Perf] speculative: Fill the DSA MTP draft's prefill KV cache without attention for all but the last token
- [Fix] [AMD] deterministic: Use one partition in the MiniMax Gluon sparse prefill
- [Fix] [AMD] deterministic: Batch-invariant channel-wise FP8 linears and experts on ROCm
- [Test] [AMD] deterministic: Check AITER paged decode batch invariance on gfx950
- [Fix] [AMD] moe: Run compressed-tensors channel-wise FP8 experts correctly on the Triton runner with AITER
- [Bug] FlashInfer backend: Triton fallback in `_safe_merge_state` treats base-2 LSE as natural log
- [Test] [NPU] HiCache: Add NPU e2e test for --hicache-host-memory-fraction
- [Bug] [SM120]DeepSeek v4.1 flash Unstable FullTopK index order in C1 and C2 decode
- Docs
- Python not yet supported