sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- Vendor eval dataset files to avoid DNS failures in CI
- WIP: Supports the VLM of GLM Image.
- Fix HF3FS batch_set_v1 missing post-processing for zero-copy MHA
- [Embedding] Support perplexity-ai/pplx-embed-v1-0.6b embedding model
- remove mamba slot alloc in prefix match stage
- [Apple Silicon] Optimize MoE performance
- [test] add registered tests for streaming ASR with consolidated ASR testing utils
- [1/N][HiCache PP]: use logical clock for deterministic LRU eviction across PP ranks
- [Bug] Mamba radix cache can turn fresh prefill prefix hits into 0-hit after split
- [HiCache & Hybrid model] Support mla + mamba hybrid model
- Docs
- Python not yet supported