sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Perf] logprob: Keep input top-k and token-ids logprobs off the host sync path
- [Doc] beam: Add deployment cookbook
- [Feat] beam: Add native Beam serving support
- [Fix] SamplingParams: validate stop_token_ids against vocab_size
- [Bug] Kimi-K3: `nvidia_kda` prefill produces NaN after ~1,800 tokens (Triton is correct)
- [Bug] MiMo-V2.6: MiMoV2Processor hard-codes CLIP mean/std, checkpoint preprocessor_config specifies ImageNet
- [Fix] benchmark: Respect explicit random dataset paths
- [Fix] scheduler: Park chunked prefill continuation when decode headroom exhausts the budget
- [Doc] blogs: Sync LMSYS SGLang blog cards
- [Bug] With `pause_generation(in_place)` then `pause_generation(abort)`, the server hangs until killed by watchdog
- Docs
- Python not yet supported