One training iteration end to end: what runs, what stays frozen and where gradients flow. Then robotics vs. LLM side by side with worked examples, and every formula.

Generic, with robotics and LLM variants

Soft Actor-Critic as used today: twin Q critics, target critics and automatic temperature, with the four updates behind every gradient step.

Robotics and continuous control

What a target network is, why the deadly triad needs one, hard vs. soft updates, and which algorithms use it.

The slow copy behind bootstrapped targets

Target Networks

The simplest policy-gradient method, step by step: sample episodes, compute returns and a baseline, take one gradient step. Includes the baselines behind RLOO, ReMax and GRPO.

RL Cheat Sheet