sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Feat] rust-server: External corpus routes for NGRAM speculative decoding
- [Feat] rust-server: /open_session, /close_session and session_params on generate
- [Feat] rust-server: configure_logging, the dumper routes, the HiCache L3 layout and elastic EP state
- [Feat] rust-server: The Ollama routes over HTTP and gRPC
- [Feat] rust-server: The Anthropic Messages API over HTTP and gRPC
- [Feat] api: The OpenAI completions and chat completions schema and gRPC surface
- [Feat] rust-server: /v1/score, and the embeddings, classify, rerank and tokenize schema
- [Feat] rust-server: Vertex and SageMaker generation wrappers, PUT /generate and slashed model ids
- [Feat] rust-server: The stateless /v1/responses API over the chat route
- [CI] [AMD] muse-spark: Add a MI35x nightly MMMU accuracy gate
- Docs
- Python not yet supported