frontier

The jagged frontier is permanent. Build like it.

In 2023, researchers at Harvard Business School gave 758 BCG consultants a set of tasks and access to a frontier model. On tasks inside the model's capability, the results were dramatic: consultants finished more work, finished it faster, and produced higher-quality output. Then the researchers added one task deliberately chosen to sit outside the model's capability. On that task, consultants using AI were 19 percentage points less likely to reach the correct answer than consultants working alone.

The model did not merely fail to help. It made trained professionals worse than they would have been without it, because it failed fluently, and fluent failure recruits trust.

That study, Dell'Acqua et al.'s "Navigating the Jagged Technological Frontier", gave the phenomenon its name. The jagged frontier is the observation that model capability is not a smooth surface. A system that writes production-grade code can miscount the items in a list. A system that passes a bar exam can lose track of which object is on top of which. Strength and weakness alternate in patterns that do not follow human intuitions about difficulty. Easy for us predicts nothing about easy for it.

Most people who encounter the idea file it as a maturity problem. The models are young, the frontier is rough, the next release will sand it down. But in reality it's a more nuanced story.

Each new model moves the frontier outward. None of them smooths it. The jag lines redraw somewhere else: a task that failed last quarter now works, and a task you had come to rely on quietly regresses. The frontier of the current generation is not a polished version of the last one. It is a different coastline entirely.

This has consequences for anyone maintaining a mental map of what AI can do. Your map has basically a shelf life of one release cycle. The effort you spend charting today's frontier is written off the moment the model weights change, and the model weights change constantly. Capability mapping is a bit of a futile exercise, a bit like a treadmill dressed up as due diligence.

A popular benchmark is Artificial Analysis' Intelligence Index. It incorporates 9 evaluations: GDPval-AA v2, 𝜏³-Banking, Terminal-Bench v2.1, SciCode, Humanity's Last Exam, GPQA Diamond, CritPt, AA-Omniscience and AA-LCRThe for a single number. However, a benchmark like this reports the average altitude of a coastline. This is useful for getting an idea how your model of choice compares with other models on average, but your workflow does not run on the average. It runs on specific steps, in sequence, and a single step below the waterline drowns the pipeline regardless of how impressive the mean is. This might be tolerable in a chat interface, where a human reads every output and the blast radius of a bad answer is one conversation. It is intolerable in agentic systems, where outputs become inputs and errors compound instead of averaging out. A 95 per cent reliable step, chained ten deep, gives you a coin flip.

The second casualty is the intuition of proximity. Humans assume that tasks which look adjacent have similar difficulty, because for humans they do. A colleague who can summarise a contract can summarise a slightly longer contract. Models break this assumption routinely. The frontier is jagged relative to our expectations of it, which means the expectations are the failure mode. Every time you extend trust from a tested task to a neighbouring untested one, you are navigating by a map the territory no longer matches.

So what is the correct response, if mapping is futile and proximity lies? The answer is: Stop trying to know where the frontier is. Build so that its location does not matter. But how do you do this?

The systems that survive jaggedness share one architectural decision: they never let generation and verification live in the same place. The model proposes; something grounded disposes. A test suite runs. A compiler objects. A schema validates. A second process with different failure modes checks the first. The model is treated as an untrusted generator inside a trusted harness, and the harness, not the model's reputation, is what earns the deployment.

This inverts the usual question. Teams ask "is the model good enough for this workflow?" and then argue about benchmarks. The better question decomposes: which steps in this workflow have a grounded check available, and which do not? Steps with grounded checks can be handed to the model today, whatever the frontier looks like, because failure is caught at the step where it happens. Steps without grounded checks are where your architecture is exposed, and no benchmark score changes that exposure. The sorting looks like this:

  1. Steps a machine can verify: automate freely. The frontier's shape is irrelevant because the check catches what the map would have missed.
  2. Steps a machine can partially verify: automate with the check in the loop, and log what slips through.
  3. Steps only a human can judge: keep the human. Not as a supervisor of everything, but as the verification layer for precisely the steps that have no other one.

Notice what this buys you. A system built this way is indifferent to releases. When the frontier redraws itself, the grounded steps keep passing or start failing visibly, and the human-judged steps were never trusting the model anyway. Nothing needs remapping because nothing depended on the map.

The uncomfortable part is that the mapping instinct dies hard. It feels responsible. It produces documents, capability matrices, approved-use lists, all the artefacts of diligence. And every one of them is stale before it circulates, quietly licensing trust in tasks the current model handles differently from the one that was tested. The diligence is the vulnerability.

The frontier will keep moving faster than any map of it. The question that matters is not where the jag lines are this month. It is this: how many steps in your own pipeline are you trusting right now on proximity alone, because a nearby task worked once, on a model that no longer exists?

Unlock the Future of Business with AI

Dive into our immersive workshops and equip your team with the tools and knowledge to lead in the AI era.

Scroll to top