I see many debates on how local LLMs will save us. This will serve as a permalink answer.

The claims:

• We're getting closer and closer to running good-enough models locally.
• A useful model will fit on our smartphones in a few years.
• We won't need to rely on Anthropic/OpenAI/China at that point.

Here's my take.

The good

Local models will indeed reach a threshold of "useful" in a few years, i.e. what we rely on for big corps today will be available locally in a few years. We'll be able to work "offline" if we want to, which is great!

The bad

OpenAI stated they're on track to have "well over" 1 million GPUs online by the end of 2025. They want to 100x that. Even if that doesn't pan out, the big corps are buying GPUs en masse. And they're not going to stop.

Furthermore, a single strong consumer GPU like the one in M5 is still 10x to 100x slower than a datacenter-class Nvidia GPU.

Even if I make mistakes in the numbers, even if the situation changes in the coming years, it doesn't matter. The sheer magnitude of difference in computing power is enormous.

What does this mean for us? It means whatever local models will be capable of, the big corps will have orders of magnitudes smarter ones.

Think of it this way: even if the local models were only 10x worse than SotA models, what will the SotA models look like when the local models will be today's SotA? No matter what you do, the big corps have a massive compute advantage, and the gap will keep widening.

GPU is the new oil.