Transformers & LLMs cheatsheet for Stanford's CME 295

Available in العربية - Čeština - English - Español - فارسی - Français - Italiano - 日本語 - 한국어 - Português - Русский - Српски - ไทย - Türkçe - 中文

Goal

This repository aims at summing up in the same place all the important concepts that are covered in Stanford's CME 295 Transformers & Large Language Models course. It includes:

• Transformers: self-attention, architecture, variants
• LLMs: prompting, finetuning (SFT, LoRA), preference tuning (RLHF, DPO), reasoning (RLVR, OPD)
• Optimizations: Distributed training, KV caching, speculative decoding
• Applications: AI agents, evaluation, extensions (e.g. diffusion LLMs)

Content

VIP Cheatsheet

Class textbook

This VIP cheatsheet gives an overview of what is in the "Super Study Guide: Transformers & Large Language Models" book, which contains ~600 illustrations over 250 pages and goes into the following concepts in depth. You can find more details at https://superstudy.guide.

Class website

cme295.stanford.edu

Authors

Afshine Amidi (Ecole Centrale Paris, MIT) and Shervine Amidi (Ecole Centrale Paris, Stanford University)