sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Feat] speculative: Report completed adaptive round execution by tier
- [Fix] grpc: preserve Prometheus metric names
- [Fix] [NPU] eplb: Preserve expert storage and reuse TP communication
- [Fix] [SAIL] utils: Query device memory without nvidia-smi
- [Bug] PD decode server fails to start with CUDA OOM on hybrid mamba models: mamba memory budget ignores decode extra slots and pre-allocated speculative states
- [Fix] mem_cache: charge PD decode pre-allocated slots in the mamba memory budget
- [Fix] hisparse: disambiguate __ffsll for HIP's dual overloads
- [Perf] kimi_k3: Compile only reachable attention-residual variants
- [Fix] benchmark: Flush one-batch benchmark output
- [Fix] sgl-model-gateway: Preserve cache_salt on chat completions
- Docs
- Python not yet supported