Aditi Raghunathan

At Carnegie Mellon, Aditi Raghunathan studies the hidden trade-offs inside modern language models: what they remember, what prolonged training can damage and how objective design constrains originality. Her work gives the field sharper tools for building systems that remain capable without becoming inscrutable.

Aditi Raghunathan works where capability, reliability and the mechanics of learning meet. An assistant professor in Carnegie Mellon University’s School of Computer Science, she asks questions that become more consequential as models grow: when does additional pretraining make adaptation harder; where is memorised information stored; and what do standard training objectives leave beyond reach? In 2025, her group contributed three unusually clear interventions. The ICML Outstanding Paper “Roll the Dice & Look Before You Leap” examined creative limits imposed by next-token prediction and argued for objectives that can explore beyond the distribution they inherit. “Overtrained Language Models Are Harder to Fine-Tune” reported that a one-billion-parameter OLMo checkpoint trained on three trillion tokens performed more than two points worse on several downstream benchmarks than an earlier 2.3-trillion-token checkpoint. “Memorization Sinks” isolated neurons associated with retained examples and reported near-perfect targeted unlearning after removing them. These are team results, with Raghunathan serving principally as a senior author. Together, they define her method: turn broad concerns about originality, adaptation and forgetting into mechanisms that can be tested, challenged and improved.