deepspeed
https://github.com/microsoft/deepspeed
Python
DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported21 Subscribers
Add a CodeTriage badge to deepspeed
Help out
- Issues
- Report the unscaled gradient norm from FP16_UnfusedOptimizer
- Decay OneCycle momentum instead of growing it past 1.0
- Check input + max_new_tokens against max_out_tokens in InferenceEngine.generate
- Run Muon once per optimizer step under ZeRO-3, not once per micro-batch
- Fix: Replace assert statements with ValueError in runtime config validation
- docs: fix two dead 1Cycle tutorial links
- docs: fix case-mismatched ZeRO tutorial links in the ZeRO-3 Offload post
- docs(megatron): fix the stale get_cuda_rng_tracker module path
- docs(comms-logging): fix the 'deepspeed.com.log_summary' module name
- docs(autotp): point the supported-models link at the readthedocs page
- Docs
- Python not yet supported