Tiny hackable CUDA language model implementation (github.com) AI

The GitHub repository “markusheimerl/gpt” describes a tiny, hackable CUDA-oriented generative transformer that models data as 8-bit byte tokens and predicts the next byte using causal self-attention, feed-forward layers, and cross-entropy loss. It outlines the model’s byte embedding, rotary positional encoding, use of AdamW and OpenBLAS for efficient matrix operations, and provides instructions and sample outputs from running an inference command.