There's a peculiar pattern everyone's noticed in the last year. The biggest, most expensive models keep getting cheaper per token while their scores on benchmarks keep climbing. Math. Code. Reasoning. The leaderboards look impressive. But ask the same models a simple factual question . who won the 1987 World Series? What's the capital of Belize? When was the Dead Sea Scrolls discovered? . and the picture changes fast. The best recall still misses half the questions on SimpleQA. The small models barely register.
I've been thinking about why this trade-off exists. And the more I look, the more it feels deliberate.
Facts take space. Procedures compress.
Research on knowledge capacity . the cleanest measurements come from the "Physics of Language Models" series . puts it on the order of two bits of factual knowledge per parameter. If you want a model that knows the birth year of every minor Wikipedia figure, the population of every Dutch municipality, and the argument order of every npm function, you pay for that in weights. It's a big part of why frontier models grew to trillions of parameters.
Reasoning compresses much better than facts. It's a small set of procedures applied over and over: break the problem into parts, track intermediate state, check your own work, backtrack when a step fails. Distillation and reinforcement learning on verifiable tasks turn out to transfer those procedures into small models remarkably well. Phi-4 is 14 billion parameters, trained heavily on synthetic textbook-style data, and it's good at math and bad at trivia. That mix used to look like a limitation of the synthetic-data approach. It now looks like the design goal.
The knowledge that survives the trade has a shape
These models are generalists. They know a little about nearly everything and almost nothing in depth. Ask one about PostgreSQL and it knows what it is, what it's good at, and roughly how MVCC works. Ask which version added a specific planner feature and you're back to invented facts. That's the right layer to keep in weights, because breadth is what lets a model understand what a question is about, know what to look up, and judge whether a source is plausible. The depth is cheap to retrieve and expensive to store, so it's the part that goes.
Facts rot, procedures don't
A frontier training run takes months and costs hundreds of millions of dollars. The moment it finishes, the facts inside it start going stale. Library APIs change, prices change, people change jobs. Half of what a 2024 model believed about the JavaScript ecosystem was outdated before the model shipped. Every fact you bake into weights has a shelf life, and the only way to refresh it is another training run.
The procedures don't rot. Algebra worked the same way in 1970 as it does now, and so does breaking a problem down or spotting a contradiction between two sources. A model that's mostly procedure and only lightly loaded with facts doesn't age the way a knowledge-heavy model does. Its training cutoff matters much less, because the current state of the world was never supposed to live in the weights in the first place.
The harness carries the knowledge
If the model doesn't know things, something else has to. That something is the retrieval: a knowledge base, tool calls, web search, a filesystem full of docs. I wrote earlier that Rust is a harness for agents, a source of cheap machine-checkable feedback. This is the same shape from the other side. The model contributes reasoning, and everything it reasons about gets supplied at runtime.
Agents don't need to memorize your dependency's API surface. They grep node_modules or read the docs before calling anything, and their answer is grounded in the version you actually have installed rather than whichever version dominated the training data. The recall that used to be a fixed cost in every forward pass became an on-demand lookup.
A frontier model on your GPU
Follow the trend a couple of years out and I think we get a model with frontier-quality reasoning that runs on a single consumer GPU. DeepSeek V4-Flash reasons with about 13 billion active parameters per token, well within consumer-GPU range. The compute half is nearly there. What doesn't fit is the other 271 billion parameters sitting in its experts, and expert layers are mostly fact storage. That's the part this whole trade makes optional. Strip the knowledge out and total size shrinks toward active size, and a 20 to 40B model at 4-bit quantization fits on the 24GB card that's been sitting in gaming PCs since 2022.
The catch is that it won't know much. Ask it a bare factual question with no tools attached and the right behavior is to say it doesn't know and go look it up. Paired with a decent harness, that's most of what I use a frontier model for today, running locally with no per-token bill and no data leaving the machine.
This mostly solves hallucination
I find this the most promising part. When a fact lives in weights, a wrong fact is unfindable and unfixable. You can't grep the weights, you can't diff them against last month, and correcting one error means a fine-tune that might break who knows what else. The model states the wrong fact with the same fluent confidence as a right one, and there's no artifact to check it against.
When the fact lives outside the model, a wrong answer has an address. The model cites a document, so you can open the document. If the document is wrong, you edit the document, and every future query gets the correction. Beats waiting for the next training run by roughly a year. Retrieval doesn't get you to zero . a model can still misread a source or stitch two of them together wrong . but a claim with a source is checkable and a claim from weights isn't. A wrong fact in a knowledge base is an ordinary data bug, the kind we already know how to trace, fix, and write a regression test for.
There's a version of this future where the model card stops listing a knowledge cutoff at all, because what's left in the weights goes stale on a scale of years instead of weeks. The model just gets handed the world's current state at runtime, the same way a CPU gets handed a program.
If that future sounds like we're all carrying a gadget that only does half the work, maybe that's the point. Half the work done well is worth more than all the work done superficially.