sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Bug] [NPU] DeepEP on Ascend A2 aborts mid decode-graph capture with a misleading EH0012 when the MC2 scratch does not fit
- [Fix] gateway: Preserve thinking budget in HTTP chat requests
- [Feat] [NPU] deepep: Add an opt-in guard for the MC2 scratch allocated during decode graph capture
- [Bug] Breakable prefill CUDA graphs return wrong, non-repeatable prompt logprobs for short prompts on Qwen3.8-2.4T-A95B-NVFP4 (B300)
- [Feature] flashinfer: Split the extend into ragged and paged passes under full prefill CUDA graphs with an FP8 KV cache
- [Feat] [NPU] attention: Support ModelSlim INT8 DSA indexer quantization
- [Doc] tests: Fix docstrings that point at test paths that do not exist
- [Fix] scheduler: Prevent pool accounting desync on finished_len trim
- [Perf] [NVIDIA] diffusion/nomas: Fuse exact head-128 mean-square reduction
- [Doc] multimodal_gen: Drop docstring params the functions do not take
- Docs
- Python not yet supported