Training Is Nothing Like Learning with Naomi Saphra (Harvard)

Naomi Saphra of Harvard's Kempner Institute argues that interpretability requires understanding training dynamics, much as biology requires evolution. Many structures found inside trained networks are vestigial: they mattered early in training but carry little meaning at the end. She describes grokking as one case of a broader pattern in which models pass through successive phase transitions, driven by symmetry breaking and attention-head specialisation, that a smooth loss curve conceals. The episode also explains why intuitions about what is hard for models often fail, for example how code in pretraining aids language reasoning and how tokenisation shapes behaviour.