llm.c
https://github.com/karpathy/llm.c
Cuda
LLM training in simple, raw C/CUDA
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Cuda not yet supported2 Subscribers
Add a CodeTriage badge to llm.c
Help out
- Issues
- fp16 buffers for ADAM
- more detailed explanation of Multi GPU
- Deleting Conda/Python as a dependency entirely to dramatically decrease "latency to step"
- Add link for SYCL runtime
- Getting "Floating point exception (core dumped)" Error
- Faster GELU forward & backward using MUFU.TANH for SM7.5+
- Broader vendor support for hardware acceleration
- [dev/cuda] Include a matmul_backward_bias kernel based on PMPP CoarsenedSumReduction kernel in 10.15
- Pretraining (with CPUs)
- [CUDA ERROR] at file \llm.c\train_gpt2.cu:405: too many resources requested for launch (old version does not have this issue - fyi)
- Docs
- Cuda not yet supported