sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Spec] Drop the unconditional stream sync from spec-prefill H2D copies
- [diffusion] Fix width/height silently dropped on multipart /v1/videos requests
- [Tests] Add regression tests for malformed nested tags and Unicode grapheme streaming
- [Fix][Profiler] Stop freezing the scheduler thread while the chrome trace is written
- Sampler: add missing TP=1 short-circuit in _sync_token_ids_across_tp
- [Kernel] Fuse W4A8 DeepEP quantization with permutation
- Disagg: enqueue middle prefill KV transfer after launch
- Support configurable Mooncake memory and NoF replica counts via environment variables
- [diffusion] Fix TeaCache CFG state lifecycle
- [MLX] Fix concurrency crashes on the MLX backend (hybrid-GDN models)
- Docs
- Python not yet supported