llm.c
https://github.com/karpathy/llm.c
Cuda
LLM training in simple, raw C/CUDA
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Cuda not yet supported2 Subscribers
Add a CodeTriage badge to llm.c
Help out
- Issues
- `make` fails to autodetect GPU compute capability
- 2D and 3D tile divisions so that permutation coordinates can be read from threadIdx and blockIdx
- Use proper GeLU on CPU
- Add cmake and fix OMP
- Recalculating the activations in the backwards pass to conserve memory
- Model Export & Inference
- Suggestion: Use smollm corpus
- [cudnn_frontend] Error: No execution plans support the graph.
- the provided PTX was compiled with an unsupported toolchain.
- The loss doesn't seem to converge after 1000 iterations
- Docs
- Cuda not yet supported