Instruction Robustness: Why Specs Written as Hints Break When You Change Models

Last week I wrote about "Spec Drift." This week we ran into a second form of the same problem.

We switched from Claude Opus 4.6 to Claude Sonnet 4.6 in the middle of a project. The commands were identical. The codebase was the same. The specifications hadn't changed. Each command included 20 mandatory sections, grep-based verification scripts, and references to gold-standard workstream documents.

The outputs looked fine at first. But one was 870 lines and the other was 111.

Opus generated an 870-line workstream that matched the gold-standard structure section by section.

Sonnet generated a 111-line workstream that looked structurally correct at first glance. The file existed, the headers were present, and nothing crashed. But most of the substance was gone: wave analysis, dependency graphs, execution strategies, and validation commands were all missing.

Neither model was wrong. The spec was fragile.

One model interpreted "the quality bar is set by this reference file" as an instruction to open and follow that file closely. The other interpreted it as contextual guidance and moved on.

The verification scripts were also present in the command, but they were advisory rather than enforced, so the agent declared the task complete without running them.

Long conversation context made the problem worse by reducing the model's thoroughness.

The fix was straightforward: change "the quality bar is set by X" to "READ X before writing and match it section for section," and make verification blocking so the pipeline fails when required sections are missing.

Section coverage went from roughly 40% to 100%.

Same command, same spec — now enforced instead of suggested.

The lesson for me was that specifications written as hints degrade across models, context windows, and time. Specifications written as constraints are much more resilient.

This isn't really about Opus versus Sonnet.

Every team deploying AI agents will eventually change models, switch providers, and upgrade versions.

The important question isn't whether a particular model follows your specification.

It's whether your specification still holds when the model underneath it changes.

That's instruction robustness.