deepspeed
https://github.com/microsoft/deepspeed
Python
DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported21 Subscribers
Add a CodeTriage badge to deepspeed
Help out
- Issues
- [REQUEST] CI: Add Ascend NPU CI pipeline
- Fix lost TP all-reduce results for non-contiguous tensors
- Fix out-of-bounds row access in inference layer_norm/rms_norm kernels for hidden sizes <= 128
- [BUG] Inference `layer_norm` / `rms_norm` return uninitialized memory for float32 inputs with hidden size 65-128
- [BUG] Inference layer_norm / rms_norm kernels read and write past the last row when rows is not a multiple of the rows per block (hidden size <= 128)
- Fix inference layer_norm/rms_norm launching no kernel for fp32 hidden sizes 65-128
- [REQUEST] Make ZeRO-1 custom-optimizer tensor flattening explicit or expose tensor metadata
- Feature Request: SuperCompress prompt compression for training throughput
- [AutoSP] Align cross entropy targets after an in-model causal label shift
- Accept warmup_num_steps=0 in the warmup LR schedulers again
- Docs
- Python not yet supported