sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Bug] `tree_speculative_sampling_target_only` rejects the only token with target mass when the coin equals the largest float32 below 1 and emits token id `vocab_size - 1`
- [Fix] Skip loading shared draft embedding and head weights
- [Bug] big prefill blocks other requests on 2xRTX3090 for qwen3.8 27b
- [cake_kernels] SP all-gather matmul route on the engine operands (view weight, any M, N % 256, capacity-bound launcher)
- [Fix] anthropic: Attach mid-turn inline system messages in place to preserve prefix cache (#42272)
- Fix zero-residual fallback in CUDA speculative sampling
- feat: add opt-in Kimi K3 CuTeDSL MLA split override
- chore: update CI test est_time values
- [Bug] /update_weights_from_disk: is_async / keep_pause / token_step have no effect, num_paused_requests is always 0
- [Bug] DeepSeek-V4 HiCache: SWA host pool is sized from the small SWA device pool, so idle prefixes miss the host tier long before the KV host pool fills
- Docs
- Python not yet supported