sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [AMD] Route MiMo V2.6 target verify through AITER MegaMoE
- [Fix] [AMD] scheduler: Keep the scheduler's GPU work on the forward stream under monokernel
- [Feat] nemotron_parse: Support Nemotron 3.5 Parse
- fix(ci): reap stale MI355X spur containers that outlive a dead runner
- [Perf] sampling: JIT softmax and argmax for the sampler and speculative verify
- [Perf] Kimi-K3: Enable the fused KDA decode at 6 and 3 heads per rank
- [Feat] http_server: Dispatch SageMaker /invocations to completions and embeddings by request body
- [Fix] tokenizer_manager: Do not dispatch a request whose abort was already sent
- [Perf] model_executor: Write small forward metadata with fill kernels behind SGLANG_ENABLE_MLP_SYNC_METADATA_FILL
- [CI] tests: Rendezvous the tile-sharding gloo test through a file store
- Docs
- Python not yet supported