llm.c
https://github.com/karpathy/llm.c
Cuda
LLM training in simple, raw C/CUDA
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Cuda not yet supported2 Subscribers
Add a CodeTriage badge to llm.c
Help out
- Issues
- [layernorm.cuh] Minor fix with 32 replaced by WARP_SIZE
- [attension.cuh] Move assert outside of attn kernel to launcher
- added reading checkpoint files
- Bugfix eval dataloader out of bound file read and crash
- Fused Forward GELU (again)
- Zero 2
- Llama RoPE Forward Kernels
- Enhance gradient norm calc in gpt2_update: reuse variables, clarify first pass logic, improve condition handling
- gpt2_forward adding CUDA streams with events for async layered operations, cache prefetching for efficient data access with high temporal locality
- WIP Distribution Visualisation to help with FP8 work & beyond
- Docs
- Cuda not yet supported