deepspeed
https://github.com/microsoft/deepspeed
Python
DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported21 Subscribers
Add a CodeTriage badge to deepspeed
Help out
- Issues
- Guard the deprecated tensor model parallel mpu fallbacks
- Warn when the Ulysses SP DataLoader is not sharded across the SP group
- Average each loss of a tuple in pipeline eval_batch
- [Triton] `is_triton_supported()` semantics for non-CUDA Triton backends
- [RFC] Vocabulary-parallel per-token ops for RL and on-policy distillation
- comms_logger: honor in-section enabled and bound per-op timer history
- [FEATURE] Cost-based automatic rollout and bucketed prefill after #8497
- [BUG] Curriculum data sampler doesn't restore its RNG on resume, so batches change after loading a checkpoint
- perf(rollout): select generation and prefill buckets with a tunable cost model
- Add opt-in TorchAO HiFloat8 Linear and AutoEP training
- Docs
- Python not yet supported