Data Curation for LLMs with Cody Blakeney (Datology AI)

Cody Blakeney of Datology AI explains why choosing training data can account for as much of a language model's final performance as major architectural advances. The episode distinguishes pre-training, mid-training and post-training data, and shows how mid-training extracts value from rare high-quality datasets and helps judge whether a multi-billion-token corpus is useful without hundreds of experimental runs. It covers filtering findings, including that short, information-dense documents often train models better than polished narrative prose, and the role and limits of synthetic data.