sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] [AMD] quantization: Keep ROCm UE8M0 group-32 FP8 quant on the v2 kernel
- [Fix] mistral_eagle: Use mm_input_embeds in the draft's prefill for multimodal requests
- [Feat] models: Public attention-layer lookup and FA kernel hooks for external models
- [Fix] serving_chat: Avoid a second BOS on multimodal text-only chats
- [Fix] tokenizer_manager: Reject input_embeds requests when speculative decoding is enabled
- [Perf] openai: Use a plain default for ChatCompletionRequest._message_tools_indices
- [Fix] [Apple] mlx: provide tree slots to auxiliary-state test fixtures
- [Perf] function_call: Avoid quadratic rescanning in MuseGlimmerDetector streaming
- [CI] [AMD] nightly: Report partial and incomplete job results
- [CI] [AMD] retry: Preserve crash and invocation failures
- Docs
- Python not yet supported