accelerate
https://github.com/huggingface/accelerate
Python
🚀 A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to accelerate
Help out
- Issues
- Fix pad_input_tensors to pad the requested dim, not always dim 0
- Fix disk offload crash on FP8 tensors
- Fix distributedType -> distributed_type typo in downcast_bf16 guard
- Fix convert_model_to_fp8_ao converting the first and last linear layers
- chore(deps): bump the actions group across 1 directory with 7 updates
- Fix moved nd_parallel.py example path in context-parallelism guide
- [docs] Add a concept guide on sharding iterable datasets
- [FSDP2] Add additional_modules_to_shard for sharding custom modules
- Add end-to-end distributed test suite (Phase 0: DDP smoke + env parity)
- [launch] Report one actionable diagnostic when distributed ranks fail (and make --quiet work for multi-GPU)
- Docs
- Python not yet supported