deepspeed
https://github.com/microsoft/deepspeed
Python
DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported15 Subscribers
Add a CodeTriage badge to deepspeed
Help out
- Issues
- [BUG] Problem with CPUAdam compilation on AMD CPUs
- [BUG] Problems when training in GH200 architecture
- [AutoTP + DS2] no memory reduction when using auto = 4
- Is there more detailed documentation for HF AutoTP training?
- Create COMMITTERS_RESPONSIBILITY.md
- [Draft] Muon Optimizer Support for ZeRO3
- Enable shm_comm support for arm
- fix: Ensure full gradient reduction for Muon with reduce_scatter
- [BUG] DeepCompile in ZeRO-1 fails to do the forward pass
- Z1/2 should flatten tensors on gpu
- Docs
- Python not yet supported