Cameron Wolfe maps RL training from REINFORCE to GRPO

Cameron Wolfe published a single document on reinforcement learning for large language models. It runs from REINFORCE to the methods behind frontier training, on his newsletter Deep Learning Focus. The throughline explains why GRPO-family training behaves the way it does, past the mechanics of the algorithm itself.

Wolfe credits the paper, textbook, or codebase behind each section, tying theory to code instead of leaving the lineage scattered across sources. RL fine-tuning hyperparameters often get set by guesswork because that lineage is hard to find in one place.

The post starts at first principles, then walks through the policy gradient algorithms behind current training runs. It moves into reasoning, agents, token efficiency, and reliability, with each section linking a longer writeup on that piece.

Wolfe names his sources directly: Nathan Lambert's RLHF Book, Sutton and Barto's textbook, and OpenAI's Spinning Up guide. He also cites Lilian Weng's notes on policy gradients and the TRL and OpenInstruct codebases.