One of my favorite parts of hosting First Principles is the opportunity to sit down with researchers whose work has fundamentally shaped the field.

Few people fit that description better than Sander Dieleman.

Sander has spent more than a decade at Google DeepMind working on some of the most influential machine learning systems of the modern era. His work spans everything from AlphaGo and WaveNet to the diffusion models that now power many of today's leading image and video generation systems.

What makes Sander particularly interesting is not just the breadth of his contributions, but his ability to explain complex ideas with unusual clarity. Through his research and widely-read technical blog, he has developed a reputation for making some of machine learning's most challenging concepts accessible without sacrificing rigor.

In our latest episode of First Principles, we explored the evolution of diffusion models, the surprising connections between different machine learning paradigms, and what happens when our intuitions break down in high-dimensional spaces.

Diffusion Isn't as Different as It Looks

One theme that emerged repeatedly throughout our conversation was that many seemingly distinct machine learning techniques are often different views of the same underlying idea.

Sander walked through the history of diffusion models, tracing their roots through score matching, denoising autoencoders, and more recent developments such as flow matching. Rather than viewing these as entirely separate approaches, he highlighted how each provides a different lens for understanding a similar generative process.

It's a perspective I find particularly valuable. Progress in machine learning is often framed as a sequence of revolutions. In practice, many breakthroughs emerge when researchers recognize deeper connections between existing ideas.

Why Diffusion Models Won

Diffusion models have become the dominant paradigm for image generation, but understanding why is more interesting than simply observing their success.

One explanation Sander offered is that diffusion breaks an incredibly difficult problem into a sequence of smaller refinement steps. Rather than generating a complete image all at once, the model gradually improves its prediction through an iterative process.

He described this as part of a broader trend in machine learning: finding clever ways to build effectively deeper computation without making optimization impossible. It's an elegant way of thinking about how many of the field's major advances fit together.

The Power of Better Explanations

Beyond the technical discussion, I wanted to understand Sander's approach to communication.

Many researchers publish papers. Far fewer consistently produce explanations that help others build intuition.

Sander described his writing process as beginning with ideas he finds himself explaining repeatedly. What starts as a short note often grows into a much larger exploration as he works through the details himself. In many cases, writing becomes a tool for learning rather than simply documenting conclusions.

That philosophy resonates strongly with me. Some of the most valuable scientific communication doesn't happen after understanding is complete; it happens during the process of developing it.

When Human Intuition Stops Working

One of my favorite parts of the conversation centered on probability, high-dimensional spaces, and why our intuitions often fail us.

As humans, we are remarkably good at reasoning about the physical world we experience directly. We are much less equipped to reason about distributions that exist in thousands or millions of dimensions.

Sander explained how concepts such as "most likely" and "most representative" can diverge dramatically in high-dimensional spaces. What appears intuitive in one dimension can become deeply misleading when scaled up.

These ideas sit at the heart of modern generative modeling, but they also highlight a broader challenge for AI research: developing mental models that help us understand systems operating far outside everyday human experience.

First Principles

The conversation covered much more than diffusion models. We also discussed music generation, the evolution of generative AI, the role of technical writing in research, and the unexpected path that led Sander from Kaggle competitions to DeepMind.

What I enjoyed most was seeing how a researcher who has contributed to so many important advances continues to think from first principles - looking for simpler explanations, deeper connections, and better ways to understand complex systems.

Watch the full conversation below.