deepspeed
https://github.com/microsoft/deepspeed
Python
DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported21 Subscribers
Add a CodeTriage badge to deepspeed
Help out
- Issues
- Fix flops profiler FLOPS and samples/s under gradient accumulation
- Round to nearest in NPUQuantizer, matching the CUDA quantizer
- Detect NEON when py-cpuinfo reports no CPU flags
- [BUG] Autotuner latency metric keeps the slowest config, and FLOPS does not match metrics.json
- Fix autotuner latency ranking and the FLOPS metric name
- Add an opt-in route_norm flag for the legacy MoE gate
- [REQUEST] Legacy MoE gate: opt-in route_norm, with historical combine weights left as the default
- AutoEP: opt-in routing weights inside the expert activation
- [BUG] OneCycle overwrites the optimizer's beta2 with 0.99
- [BUG] OneCycle.get_last_lr() raises before the first step() while other schedules return the initial lr
- Docs
- Python not yet supported