accelerate
https://github.com/huggingface/accelerate
Python
🚀 A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to accelerate
Help out
- Issues
- FSDP2 RAM-efficient loading fails on persistent buffers with AttributeError: 'Tensor' object has no attribute 'device_mesh'
- Preserve empty tensors when saving safetensors
- Restore environment variables when patched keys differ only in case
- Add BF16/FP8 training benchmarks and refresh the TE container
- Preserve dictionary inputs in split_between_processes
- Handle empty tensors in disk offload
- Update Transformer Engine integration to current public APIs
- Fix DeepSpeed auto warmup LR for multiple optimizer groups
- Fix sampler handling in skip_first_batches
- Clean up the files of a previous save in `Accelerator.save_model` for safetensors
- Docs
- Python not yet supported