sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] Accept pack-quantized (WNA16) lm_head in should_apply_lm_head_quant_method so DFLASH does not crash
- Fuse MiMo verification RoPE and KV cache writes
- [Feature] Add draft prefetch on Eagle3
- feat(jit): support native MUSA compilation
- [Bug] `--enable-deterministic-inference` + one request with `repetition_penalty` kills the scheduler on granite-4.0-h (torch.compile `InternalTorchDynamoError` in `apply_scaling_penalties`)
- [Build][CI] Drop Python 3.10: require Python 3.11+ and run CUDA CI on Python 3.12
- [Spec] Fix DFLASH verify mask policy for wrapped attention backends
- feat(elastic-ep): add fixed-slot recovery admission and warmup lifecycle
- [Inkling] Render and parse chat through tml-renderers
- [Spec] Sample from the target when the rejection residual is empty in TreeSpeculativeSamplingTargetOnly
- Docs
- Python not yet supported