deepspeed
https://github.com/microsoft/deepspeed
Python
DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported21 Subscribers
Add a CodeTriage badge to deepspeed
Help out
- Issues
- Fix NameError in unwrap_model_for_generation (optimizer=None) and dead rank= in allreduce_bucket
- Guard steps_per_print() None check in PipelineEngine._exec_optimizer_step
- Log a warning when ZeRO gradient overflow silently skips the optimizer step
- Fix SequenceTiledCompute backward for empty trailing shards
- Validate max_out_tokens and min_out_tokens are positive at inference config parse time
- ZeRO-3: adaptive prefetch bucket size
- `_cast_module_mixed_precision` casts every dtype torch calls floating point, corrupting FP8/MX/NVFP4 parameters
- ZeRO skips the optimizer step on gradient overflow without logging anything
- Fix DSVAE's forward path and the arguments it drops
- DeepCompile: reuse ZeRO-3 all-gather memory
- Docs
- Python not yet supported