sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Spec] Deduplicate the max_running_requests=48 default into one helper
- [AMD] Kimi-K3 MoE front optimization
- fix(multimodal): let load_video accept an already-constructed decoder
- fix(multimodal): resolve image token counts for decoded image tensors
- [Speculative] Derive the DSpark verify-budget cost table at CUDA graph capture
- [Docs]: clarify chunked_prefill_size and max_prefill_tokens semantics
- [Bug] MiniMax-H3 FL2VA + Turbo LoRA: `slice_lora_b_weights` crashes on merged/QKV linear layers with TP>1
- [Kernel] Use Cake source Router GEMM for DeepSeek, GLM, and Mistral
- [Feature] Add an opt-in SM80 fallback for DeepSeek Sparse Attention (DSA) on NVIDIA A100
- [NPU] Reduce graph replay host overhead
- Docs
- Python not yet supported