sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] glm47: Strip the function name in detect_and_parse so a trailing newline does not drop the call
- [Feat] serving: Add --clamp-max-new-tokens-to-context to cap max_tokens when the prompt fits
- [Feat] moe: Let models opt out of the partial-block quantized MoE shard check
- [Perf] mem_cache: build non-Triton extend bookkeeping on the host
- [Bug] HiCache `buffer_only` + Mooncake L3: scheduler admits a chunked prefill it cannot finish, then crashes with "Prefill out of memory" (evictable 0, every radix node locked)
- [Doc] supported-models: Fix test suite path and command in support_new_models
- [Feat] simulator: support AISimulate FPM estimation modes
- [Bug][function_call] GLM tool-call parsers turn JSON null into the string "None"/"null"
- [Doc] diffusion: validate convrot_int8 on Wan2.2 A14B (T2V & I2V), Wan-Animate-2 (base & distilled)
- [Doc] diffusion: Fix two documented imports that raise ImportError
- Docs
- Python not yet supported