sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [PD] Let Rust-server prefill with multiple DP ranks register its DP rank
- [Rust] Warm up every attention-DP listener so each reports healthy
- [cherrypick from #42349] [RL] Pass return_indexer_topk through chat completions and sessions
- [cherrypick from #42352] [RL] Return replay outputs through a Mooncake output store
- [Bug] PP + EAGLE/MTP spec decoding crashes with --enable-mixed-chunk: 'EagleDraftInput' object has no attribute 'rids' in _pp_spec_rebuild_verify_input
- [Mamba2] Add checkpoint-preserving FlashInfer SSD prefill
- [Bug] `--enable-unified-memory` ignores `--max-total-tokens` on hybrid-Mamba models
- [Bug] `--enable-unified-memory` on a hybrid-SWA model kills the scheduler, `alloc_token_slots` raises "Out of memory"
- [MoE] Enable deferred finalize for NVFP4 on the routed TRT-LLM runner
- [LoRA] Reject per-module rank and alpha overrides in PEFT adapters
- Docs
- Python not yet supported