You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
160+ pages of handwritten notes from Stanford CS336: Language Modeling from Scratch, covering language modeling, LLM systems, and full-stack AI engineering from data to deployment and evaluation and RL alignment
Implementation of a Decoder-only Transformer language model from scratch for CS336, featuring a byte-level BPE tokenizer, RoPE, Multi-Head Self-Attention and SwiGLU FFN. Trained on TinyStories with 1.39 Val Loss.
Living comparison table of LLM architectural choices (norm, attention, MoE, positional encoding, and more) from the Original Transformer (2017) to frontier models (2026). Based on Harm de Vries's figure, Sebastian Raschka's Big LLM Architecture Comparison, and Tatsunori Hashimoto's Stanford CS 336 lecture.