sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] deterministic: Align chunked prefill to the Gated DeltaNet block size
- [HiCache] Fix and document HF3FS benchmarks
- (draft) [Fix] [AMD] quark: Load FP8 shared experts into MXFP4 fused MoE weights
- [Fix] PD: Skip the cached-prefix early send in batches that start a HiCache load-back
- [Fix] comfyui: Fix server-mode bugs in the SGLDiffusion plugin
- [Bug] [diffusion] ComfyUI plugin: server-mode bugs in SGLDiffusion nodes
- [Feature] [diffusion] ComfyUI plugin: free the SGLD worker's GPU memory when ComfyUI unloads the model
- [Feat] comfyui: Free the SGLD worker's GPU memory when ComfyUI unloads the model
- [Bug] Ordinary --api-key authorizes ADMIN_OPTIONAL management endpoints when --admin-api-key is unset
- [Bug] NanoNemotron globally disables PIL MAX_IMAGE_PIXELS and removes decompression-bomb protection
- Docs
- Python not yet supported