sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Model] Apply Gemma4 attention overrides to text-only checkpoints
- [Perf] Fuse asynchronous NaN validation into one kernel
- [Bug] NGRAM speculative decoding over-samples token id 0 at temperature > 0 (zero-padded draft tree)
- [DSv4] Size the sparse prefill workspace per request, not num_reqs x longest
- [HiCache] Export host Mamba pool occupancy next to host KV
- [Anthropic] Add opt-in keepalives for idle message streams
- [Function Call] Avoid copying already streamed GLM47 argument values
- [Bug][ROCm] NGRAM speculative decoding on a DSA model (GLM-5.3, gfx942): HSA 0x1016 on every rank after the first ~50k-token prompt
- [Bug] [Diffusion] Concurrent instances: scheduler silently rebinds to port+42 and requests reach the other instance
- [Diffusion] Keep ZMQ ingress sockets on their configured port
- Docs
- Python not yet supported