Here is the test for whether your AI architecture belongs to you. If the primary model vendor doubles the price or shuts the door tomorrow, can you swap the weights without losing the operational edge?
If the answer is no, you do not own a capability. You have a rented generalist sitting in an executive chair.
Most teams treat the foundation model as the firm's intellectual property. They refine prompts, attach tools, and wait for the next frontier release to clean up the edge cases. When the base model changes, domain performance falls back to whatever a stranger with no memory of your work can do on day one. That is lock-in wearing an engineering badge.
In an earlier post I argued that the moat moved to the runtime. You can own a great harness and still lose if you do not own the gradeable loop. A harness without that loop is empty plumbing around someone else's weights.
Satya's swap test is the cleanest statement of this I have seen. A company should be able to switch out the generalist and keep the veteran. The veteran is not a prompt file. It is private evals against your outcomes, traces you can grade, a private reinforcement-learning environment that trains on those traces, and institutional memory you can query. He calls the whole thing a hill-climbing machine, and treats it as the new IP of the firm.
Traces, evals, adapted weights, and memory accumulate in one place. That place is the asset. If it sits on the vendor's side of the wall, you are training their veteran. If it sits on yours, the intelligence of the system compounds on your balance sheet, and the generalist becomes interchangeable.
We have known the loop can beat the bigger base model since 2022. After InstructGPT, human labelers preferred a 1.3B model to 175B GPT-3, at least on that paper's prompts. The step change came from feedback against a defined intent, not from another hundred billion parameters.
The reason most firms never get this loop is not missing tooling. Most real work is still not easy to turn into a trace you can score. You can watch a human take a golden path and still not know whether the counterfactual paths were good or bad. Andrew Ho put that bottleneck plainly: economically useful work is highly contextual, and encoding it so a model can be scored is the hard part. If live completions never become a trace you can score, private RL has nothing to chew on. Memory has nothing durable to store. You stay on public benchmarks, which measure a generalist, not a veteran.
I am not claiming you can skip the foundation model. You still need frontier reasoning to bootstrap hard tasks and to generate the first useful traces.
I am not claiming harnesses are worthless. You still need runtime execution, guardrails, and tool sandboxes to ship anything reliable.
To build a veteran that survives the next model migration, do this in order of retrofit cost.
Instrument live workflows so every completion emits a trace you can score. That is mostly logging and a grader contract, and it is the cheapest move that makes the rest possible.
Stand up private evals against your outcomes. Not the public leaderboard. The tickets you close, the refunds you avoid, the reviews that do not come back.
Then the expensive step: a loop from traces into RL and memory, before the next model bake-off. If you only start this work when a new model ships, you will spend the bake-off relearning what you already knew.
Then fire the generalist. Swap the base model. If the veteran still works, you own the asset. If it does not, you were renting the chair.