deepspeed
https://github.com/microsoft/deepspeed
Python
DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported21 Subscribers
Add a CodeTriage badge to deepspeed
Help out
- Issues
- [AutoSP] Fix compiler correctness
- AutoTP loads module weights by hardcoded class name, so 219 of 233 transformers norm classes are silently left on meta
- fix(zenflow): restore assert and remove duplicated identical branch in gradient copy
- fix(checkpointing): remove identical if/else arms and leftover debug print in WriterFactory
- DeepSpeed cannot start without mpi4py on a machine with no launcher
- Muon under AutoTP orthogonalizes each shard, so the update depends on the TP degree
- Default gradient_clipping divides every Muon update by its own norm, shrinking the step by a model-sized factor
- ZeRO-3 applies Muon's Newton-Schulz once per micro-batch, so gradient accumulation changes the optimizer
- Fix curriculum sampler checkpointing global np.random instead of its own np_rng
- Fix missing () on is_bf16_supported in ZeRO-3 dtype fallback (silent bf16 on non-bf16 hardware)
- Docs
- Python not yet supported