sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Bug] deepep FP8 MoE on GB300/sm_103 crashes in DeepGEMM m_grouped_fp8_fp4_gemm_nt_contiguous — UE8M0 scale-format mismatch (CUDA 719 / smxx_layout.cuh:131 assert)
- [Kernel] Tune Kimi-K3 Marlin MoE block sizes on H200
- model_loader: only open the MTP-head shards for a NextN/MTP draft load
- [Spec] Deduplicate the max_running_requests=48 default into one helper
- [AMD] Kimi-K3 MoE front optimization
- fix(multimodal): let load_video accept an already-constructed decoder
- fix(multimodal): resolve image token counts for decoded image tensors
- [Speculative] Derive the DSpark verify-budget cost table at CUDA graph capture
- [Docs]: clarify chunked_prefill_size and max_prefill_tokens semantics
- [Bug] MiniMax-H3 FL2VA + Turbo LoRA: `slice_lora_b_weights` crashes on merged/QKV linear layers with TP>1
- Docs
- Python not yet supported