deepspeed
https://github.com/microsoft/deepspeed
Python
DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported21 Subscribers
Add a CodeTriage badge to deepspeed
Help out
- Issues
- Fixed memory leak in loss backward with AC & zero3 & single-model-multi-branch network
- docs: fix precompiled ops install command
- Deprecate unused DeepSpeed features
- [RFC] Enable Correct and High-Performance Muon Optimizer under ZeRO CPU Offload (Stage 1/2/3)
- Make add_tuning_arguments' 1Cycle flags reach OneCycle
- A zero-element parameter loses its shape when it is bound to a flat buffer
- fix: track FusedAdam bias correction per parameter
- Fix Muon optimizer under ZeRO CPU offload and bound gather buffers
- fix(inference): break circular import in ops.transformer.inference
- Support dynamic parameter offload for ZeRO-3 inference
- Docs
- Python not yet supported