deepspeed
https://github.com/microsoft/deepspeed
Python
DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported21 Subscribers
Add a CodeTriage badge to deepspeed
Help out
- Issues
- [rollout] Affine weight-sync building block: ZeRO storage into a tensor-parallel inference copy, shard to shard
- Avoid quadratic copies in curriculum cluster selection
- nightly failure without a failure class: cpu-torch-multi-latest died unclassified
- [AffineIR] Check strided shard destinations without pairwise comparisons
- Fix oversized zero_to_fp32 shards for ZeRO-1/2 checkpoints
- klxpu: enable cuda graph capture
- ZeRO-2 CPU offload: accumulate in the compact partition between requests
- [HybridEngineRollout] Add multi-turn tool-calling rollout, GRPO primitives and turn-level continuous batching
- nightly regression: cpu-torch-multi-latest multi-rank suite failed
- Release the GIL during native CPU Adam updates
- Docs
- Python not yet supported