accelerate
https://github.com/huggingface/accelerate
Python
🚀 A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to accelerate
Help out
- Issues
- Raise ValueError in find_batch_size for 0-dim tensors
- load_checkpoint_and_dispatch ignores dtype with default device_map=None
- Count tied partner modules once in device-map sizing
- Forward epochs to sharded custom batch samplers
- Remove stale alternate-format weights after checkpoint saves
- `preserve_rng_stage=False` in fsdp2_apply_ac causes bug when using dropout with activation checkpointing
- Allow skipping IterableDataset re-sharding via shard_iterable_dataset
- Fix `get_balanced_memory` raising on a "disk" key in `max_memory`
- Run the FSDP2 forward under mixed precision autocast, as for DDP and FSDP1
- FSDP2 forward does not run under mixed precision autocast, unlike DDP and FSDP1
- Docs
- Python not yet supported