llm.c
https://github.com/karpathy/llm.c
Cuda
LLM training in simple, raw C/CUDA
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Cuda not yet supported2 Subscribers
Add a CodeTriage badge to llm.c
Help out
- Issues
- Added FlameGraphs for nsys reports and some nsys documentation
- What would be the main design trade-offs when re-implementing in clean modern C++?
- Is there a plan to support 8bits (FP8 or INT8)?
- only save missing bits to reconstruct fp32 master weights
- BitNet (b1.58) support
- Realtime training visualization using wandb
- Added packed layernorm_forward
- is max_seq_len configurable or hardcoded parameter?
- RMSNorm kernels
- Adds dev container to speed up project onboarding
- Docs
- Cuda not yet supported