Compute Trends in Machine Learning

We’ve all watched machine learning progress with a mix of awe and anxiety, but Jaime Sevilla’s analysis of compute trends grounds that progress in hard, quantitative data. Looking at the sheer growth of compute inputs over time makes it clear that we’ve been running an brute-force experiment on scaling. While it's easy to get swept up in the high-level narratives of intelligence, the underlying fuel has been an aggressive, exponential expansion of floating-point operations. It makes me ask: are we building genuinely better architectures, or are we just throwing more fuel into a massive, inefficient furnace?

As a startup founder, this historical context is a sobering reality check on defensibility. If the major breakthroughs of the last decade have been primarily unlocked by scaling compute, then the game belongs to those with the deepest pockets and the most servers. For us to compete, we have to look for leverage elsewhere. We must focus on algorithmic efficiency, higher-quality data, and finding clever ways to do more with less compute. Sevilla's data shows that the trajectory is unsustainable for smaller players, which means our survival depends on being smarter, not just bigger.

What stuck with me

• Exponential scaling reality: The rapid progress of modern machine learning is overwhelmingly driven by massive, exponential increases in compute inputs.
• Resource concentration risk: Because compute scaling requires immense resources, breakthroughs are increasingly concentrated in a small group of extremely well-funded organizations.
• The efficiency imperative: Smaller companies and startups must pivot toward efficiency and alternative techniques to survive in a landscape dominated by brute-force scaling.