sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Bug] `--enable-symm-mem`: the 4 GiB symmetric-memory pool and NCCL buffers are not taken out of `--mem-fraction-static`, so the server runs out of memory after start-up
- [Perf] [NVIDIA] qsa: Keep sparse decode scratch in FP8 for FP8 KV pools
- [Bug] Native gRPC Generate caps output at 128 tokens when max_new_tokens is unset
- [Bug] Dumper filter eval() enables scheduler-process RCE
- [Bug] Mistral streaming deletes closing brackets from ordinary text
- [Bug] BF16 RL weight update with non-routed `moe_runner_backend=flashinfer_trtllm` crashes in `_load_w13` with 64-vs-2048 shape mismatch (blocked/padded w13 layout not restored on the update_weights_from_tensor path)
- [Fix] [XPU] nightly: Fix rerank, zimage, flux2 and EAGLE parity
- [Perf] glm5: Prewarm mHC prenorm kernels
- [Fix] glm47: Bound tool argument pairs
- Pass uniform_q_len to the FlashInfer target-verify plan for EAGLE speculative decoding
- Docs
- Python not yet supported