deepspeed
https://github.com/microsoft/deepspeed
Python
DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported21 Subscribers
Add a CodeTriage badge to deepspeed
Help out
- Issues
- Restore the data sampler's own RNG when resuming from a checkpoint
- Keep single-rank barrier() free of torch's device_id warning
- Skip SuperOffload's eager CPU step on non-finite gradients
- Average sparse gradients by world size regardless of gradient_predivide_factor
- ZenFlow CPU affinity split gives ranks empty core sets on Slurm allocations and multi-node runs
- Fix ZenFlow CPU affinity on Slurm masks and multi-node runs
- Single-rank init_distributed binds no device_id, so torch warns at barrier() (since 0.19.7)
- [RFC] Backend-pluggable low-precision training for Dense Linear and AutoEP grouped GEMM (Ascend HiFloat8)
- Add an opt-in attention-output stash for Hugging Face reentrant checkpointing
- [REQUEST] Inference V2: add one model family on the existing kernels (dense Qwen3 proposed)
- Docs
- Python not yet supported