llm.c
https://github.com/karpathy/llm.c
Cuda
LLM training in simple, raw C/CUDA
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Cuda not yet supported2 Subscribers
Add a CodeTriage badge to llm.c
Help out
- Issues
- Outlier detection: catch more outliers by not updating moving average with skipped updates
- Add RoPE positional encoding
- Add SwiGLU support
- TypeError: normal_() got an unexpected keyword argument 'generator'
- multi-threaded model initialization
- [WIP] initial curand implementation for model init
- Can't train in FP16 on Turing
- Add RoPE positional encoding - llama3 feature branch
- Makefile incorrectly finds that `nccl` is installed for Linux systems with `libvncclclient`
- fix: false-positive check for nccl install on ubuntu
- Docs
- Cuda not yet supported