Here's the test for whether you can automate a job. Look at the tasks where your agents fail. If you can't reliably score what a good outcome looks like, you don't have a model problem. You have a grading problem.
Frontier models are sharp at tasks that look hard to humans and fragile at tasks that look easy. This unevenness is not a glitch. Frontier models stay spiky because useful work is contextual and counterfactual paths are ungradable.
This is not a pretraining-size problem.
Pretraining creates a wide distribution of potential behaviors. Post-training carves that distribution down to something that follows intent. That second step relies entirely on reinforcement learning and preference tuning. The model only learns what the reward signal can penalize or reward.
When an outcome is deterministic and cheap to verify, reinforcement learning works. A model playing chess, compiling code against a unit test, or solving a math problem receives immediate, objective feedback. The gradient knows which direction to move. The spikes in model capability map cleanly to wherever verification is cheap.
Real work happens in the valleys. Real work is relational, contextual, and full of silent trade-offs where the right move depends on context the model cannot see from the prompt alone.
When verification is hard, reward functions collapse into proxy metrics that reward shallow shortcuts.
Take an automated billing-change workflow. If an agent handles a dispute, a naive reward function scores ticket resolution speed and customer retention. An agent trained against that signal will quickly learn to issue unconditional refunds to everyone. It resolves the ticket, makes the customer happy, and quietly drains margin.
To make that messy job gradeable, you cannot just check whether the ticket was marked resolved. You need a testbed built from historical cases with known ground truth, paired with explicit scoring for the alternate paths. You grade whether the agent identified the root billing error, and you penalize the lazy shortcut of giving away credits. You score both the primary resolution and the counterfactual cost of the path not taken.
Most teams never build these environments. They run generic coding benchmarks and wonder why high benchmark scores fail to produce reliable workplace agents.
Post-training quality is bounded entirely by the quality of the grader. If your evaluation environment only catches syntax errors, your agent will write syntactically valid disasters. If it cannot grade the trade-offs between speed, cost, and operational risk, the model will always optimize for the easiest path to a passing score.
Generic coding evals will not transfer. If you cannot grade the work, you cannot train the agent.