llm.c
https://github.com/karpathy/llm.c
Cuda
LLM training in simple, raw C/CUDA
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Cuda not yet supported2 Subscribers
Add a CodeTriage badge to llm.c
Help out
- Issues
- Speedup CPU training by 10% using Memory Aligned Tensors
- Optionally specify root location of CUDNN
- Async optimizer state and model checkpointing
- block-level stable adamw
- Store RNG seed instead of shuffled indices
- Add option to remove biases
- Recompute mlp
- FP8 work in progress
- demo how to track activations without too much boilerplate code
- Update README.md with prerequisite of libomp
- Docs
- Cuda not yet supported