llm.c
https://github.com/karpathy/llm.c
Cuda
LLM training in simple, raw C/CUDA
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Cuda not yet supported2 Subscribers
Add a CodeTriage badge to llm.c
Help out
- Issues
- Token out of vocabulary at train_gpt2.cu:675
- write LLVM optimization passes for train_gpt2
- Speedup `attention_forward_kernel2` by implementing Flash Attention 2 kernel
- looking forward supporting winx86-msvc
- Fixed a TODO to calculate the max value neatly and use inv sum trick
- [CUDA ERROR] at file train_gpt2.cu:693: out of memory
- Pytorch Profiler
- Input token length question
- delete use of cooperative groups in kernels
- Updated adamw to use packed data types
- Docs
- Cuda not yet supported