Context

This page is a worked comparison in the tradition of pbc-4445 and pbc-4849. The same Darmok scenario — This object step definition parameter set doesn't exist validation — ran on sheep-dog-grammar on consecutive nights, Rebuild50 (May 24 → 01:21 EDT on May 25) and Rebuild51 (May 25 → 11:46 EDT on May 25). The pair-range was the widest on the gsheet (gid 1236418776) for this pair across all 12 active scenarios. The diagnosis returns to pbc-4445 case 1’s signature — different files patched, both runs pass — on the same scenario that pbc-4849 tracked to a prompt-template cause and that pbc-4445 case 2 saw converge on the same files.

	Case
Scenario	`This object step definition parameter set doesn't exist validation`
Rebuild50 commit	`6decf5dd`
Rebuild51 commit	`790751de`
Green-phase range	2.79 min (166,869 ms)
Files patched	different between branches (TestStepIssueDetector vs RowIssueDetector)
Diagnosed cause	prompt-reference gap — `identify.md` cites `${umlDir}/uml-interaction.md` which doesn’t exist; the explicit `RowIssueDetector.validateCellListWorkspace` example lives in `uml-interaction-main.md`, which neither run opened

This was the widest green-phase range on the gsheet for the Rebuild50–51 pair (range 166,869 ms; next-widest was Header row Cell names at 100,207 ms). The same scenario topped the previous pair’s range in pbc-4849 at 121,761 ms. Three consecutive case studies on this scenario — pbc-4445 case 2 (same files), pbc-4849 (same files), pbc-5051 (different files) — show the scenario sits on a spec-ambiguity boundary that fires intermittently.

What We Observed

	Rebuild50 (May 25 01:21 EDT)	Rebuild51 (May 25 11:46 EDT)
Commit	`6decf5dd`	`790751de`
Phase total	817,102 ms	716,534 ms
Red phase	81,060 ms	77,284 ms
Green phase	473,823 ms	306,954 ms
Refactor phase	172,372 ms	249,709 ms
Files patched (green commit)	`TestStepIssueDetector.java` (+44), `TestStepIssueTypes.java` (+5), `ValidateActionImpl.java` (+16)	`RowIssueDetector.java` (new, +68), `RowIssueTypes.java` (new, +12), `ValidateActionImpl.java` (+10)
Tests pass after green	yes	yes
Green mvn-verify cycles	2 (first failed, second passed)	1 (passed first try)

Green-phase pair-range = 2.79 min on a 5–8 min baseline for this scenario across the rebuild series. This study scopes to green only; refactor variance is real but is treated separately.

Both runs used --model sonnet. The session UUIDs were 14521c51-a5cf-40bd-811e-3051d26c31c8 (R50) and 8b0079f8-5db7-435d-ba9c-e652a4fcdbcd (R51); neither appears in the VSCode session picker because headless claude --print invocations don’t emit the file-history-snapshot event the picker filters on — look them up by UUID from the runner log.

Where The Time Went

The session JSONLs were sliced to the green-phase window (R50: 05:21:12Z–05:28:36Z; R51: 15:46:07Z–15:50:49Z) and counts extracted.

	R50 green window	R51 green window
Wall-clock	7:24 (444 s)	4:42 (282 s)
Tool calls (total)	54	45
Grep	15	4
Read	13	13
TodoWrite	11	12
Glob	6	10
Edit	6	2
Write (new files)	0	2
Bash (mvn verify)	2	1
ToolSearch	1	1

Two pivots stand out. R50 ran 15 Greps to R51’s 4 — keyword-driven exploration versus structural discovery — and R50 needed a second mvn verify cycle after its first edits failed compile/test. R51’s path was cleaner: more Globs to enumerate files structurally, fewer Greps, only one mvn verify because its first edits stuck.

The 15 vs 4 Grep calls

Both sessions opened identically with the same five calls — ToolSearch (TodoWrite schema), two TodoWrite seeds, the COMPILATION ERROR|FAILED|ERROR|... grep on log.txt, then another TodoWrite. Divergence is at call #6.

#	R50 (15 Greps total)	R51 (4 Greps total)
6	`Read uml-interaction.md`	`Glob target/site/jacoco-with-tests/`
7	`Grep "doesn't exist"`	`TodoWrite`
8	`Glob **/uml-interaction.md`	`Read uml-interaction.md`
9	`Read TestStep*.java`	`Glob site/uml/*.md`
10	`Grep "step parameters don't exist"`	`Read uml-interaction-test.md`
11	`Grep "object step definition parameter set doesn't exist"`	`Grep "doesn't exist validation"`
12	`Read cucumber feature`	`Read cucumber feature`
13	`Grep "TEST_STEP_STEP_DEFINITION_NAME_WORKSPACE\\|validateAnnotation\\|..."`	`Glob src/test/java/.../impl/`
14	`Grep "step definition doesn't exist\\|step object.*doesn't exist\\|WORKSPACE"`	`Read ValidateActionImpl.java`
15	`Read src/main/...`	`Glob src/main/java/...`
16	`Glob */uml/.md`	`Glob src/main/java/...`
17	`Grep "step.parameter\\|parameter.set\\|validateStepParameter\\|STEP_PARAMETER"`	`Glob src/main/java/...`
18	`Grep "step.parameter\\|parameter.set\\|validateStepParameter\\|STEP_PARAMETER"` (repeat)	`Read TestStepIssueDetector.java`
19	`Read uml-interaction-test.md`	`Glob src/main/...`
20	`Glob */impl/Validate*Impl.java`	`Read TestStepIssueTypes.java`
21	`Glob */impl/Action*Impl.java`	`Glob src-gen/main/*/.java`
22	`Read src/test/.../impl/`	`Read src-gen/...IssueDetector.java`
23	`Grep "getStepParametersList\\|IStepParameters\\|stepParameters"`	`Read src-gen/...IssueTypes.java`
24	`Grep "getTable\\|getRowList\\|getCellList\\|ITable\\|IRow"`	`Read ValidateActionImpl.java`
25	`Grep "interface IStepDefinition\\|interface ITestStep\\|..."`	`Read src-gen/...`
26	`Glob **/grammar/ITestStep.java`	`Grep "RowIssue\\|TestStep.Detector\\|Row.Detector"`
27	`Glob **/ITestStep.java`	`Read src-gen/main/...`

R50 hammered on step.*parameter|STEP_PARAMETER keyword variants (calls #17, #18 identical) trying to find an existing Step-Parameters-checking code path inside TestStepIssueDetector. R51 globbed the package family, read the interface hierarchy, then grep #26 asked the disambiguating question directly: RowIssue|TestStep.*Detector|Row.*Detector — discovered neither RowIssueDetector nor RowIssueTypes existed yet, and decided to create them.

R50 spent its 4 extra minutes building a fix inside an existing class; R51 spent its identify time deciding a new class was needed and built it from scratch.

Root Cause

The diagnosis is prompt-driven anchoring on the wrong abstraction, not spec ambiguity. A Cucumber test cannot specify which implementation site a fix lives at — that’s by design, the test asserts on the observable diagnostic only. The constraint that does specify it lives in the UML interaction spec, and it’s not being read.

The convention is documented

site/uml/uml-interaction-main.md lines 92–149 explicitly specify both halves of the answer R51 reached by inference:

Lines 92–110 (## {Type}IssueDetector section) — the framework convention is one detector per cursor type. The ValidateActionImpl cascade is shown as an instanceof ITestStep branch dispatching to TestStepIssueDetector.validate{StepObject,StepDefinition}Name{Only,Workspace}. By the section’s framing, an IRow cursor demands a RowIssueDetector — the {Type} placeholder maps to whatever model class the cursor is.
Lines 143–149 spell out RowIssueDetector.validateCellListWorkspace by name, with a verbatim code example showing the exact ancestor-navigation chain (getAncestor(theRow, ITable.class) → getAncestor(table, ITestStep.class)) and the getCellListAsString comparison. This is what R51 reinvented from scratch.
Lines 129–138 also show RowIssueResolver.correctCellListWorkspace (the quick-fix half), confirming the IRow-owns-this-concern pattern across detector + resolver.

So the answer to “which detector should validate a row-cell mismatch?” is unambiguously specified: RowIssueDetector, by class name, with a working code template.

The prompt doesn’t point at the right file

identify.md’s Inputs section lists **UML interaction spec**: ${umlDir}/uml-interaction.md — but that file does not exist on disk. The actual files are uml-interaction-main.md and uml-interaction-test.md (split by production-code patterns vs test patterns). When claude attempts the Read, the path 404s. From there:

R50 (call #6) attempted uml-interaction.md, got nothing, eventually globbed and read uml-interaction-test.md (call #19) — but never read uml-interaction-main.md where the answer lives. With no spec guidance, R50 fell back to pattern-matching the method list it could see inside TestStepIssueDetector.java and concluded the missing 5th method belonged on that class.
R51 (call #8) attempted the same 404, also read only uml-interaction-test.md (call #10), never read uml-interaction-main.md. R51’s correct conclusion came from a separate convention it inferred by reading ValidateActionImpl.java (call #14) before opening any detector class — the cursor instanceof branches read like a one-detector-per-type dispatch table, and the absence of an instanceof IRow branch combined with the failing scenario’s rowList/1 cursor path made the answer fall out.

R51 was right by luck of read-ordering, not because the spec told it. Run it again with a different read sequence and R51 could just as easily land on R50’s path.

Earlier same-scenario pairs converged because the codebase didn’t tempt them

This is the same scenario pbc-4445 case 2 and pbc-4849 both tracked. Those earlier pairs converged on the Row* path because either TestStepIssueDetector didn’t yet have surface area that made the validateStepParameters extension plausible, or the model in those runs happened to read in R51’s order. Either way, the underlying gap — the prompt failing to land claude on uml-interaction-main.md — has been latent across every prior case study on this scenario. It surfaced now because the pattern-matching coin landed differently.

The Fix

The fix sits in the prompt + UML file layout, not in the test (which by construction can’t specify implementation choice). Three reachable changes, ordered by cost:

Fix identify.md’s UML reference path. Change ${umlDir}/uml-interaction.md to ${umlDir}/uml-interaction-main.md (the file the answer actually lives in), or glob the directory (${umlDir}/uml-interaction*.md) so claude reads both halves. Smallest change; closes the immediate gap. Would require a corresponding fix to fix.md if it also cites the singular path.
Rename / consolidate the UML interaction files so uml-interaction.md exists. Options: (a) rename uml-interaction-main.md → uml-interaction.md and uml-interaction-test.md stays the same; (b) merge both into a single uml-interaction.md. Either way the prompt’s existing path resolves. (b) is simpler but loses the main-vs-test split that has its own value.
Add an explicit ordering hint in identify.md. “Before opening any {Type}IssueDetector class, read the UML interaction spec’s {Type}IssueDetector section AND read ValidateActionImpl (or equivalent action-dispatch entry point) to identify which model node type owns the new check.” Belt-and-braces — works even if the file rename happens later.

Pbc-4849’s recommended fix (precompute JaCoCo shortlist, issue 404) addressed a different gap — which classes to study for prior-art templates. Pbc-5051’s gap is upstream: which spec file tells claude where the check belongs in the first place. (1) and (2) are independent of 404 and complement it.

A follow-up question worth asking: why didn’t the refactor phase catch the divergence? Refactor reads the UML class-level specs (uml-class-*.md) and is expected to flag deviations from the documented pattern. If RowIssueDetector is documented in uml-class-*.md and refactor didn’t push R50’s TestStepIssueDetector deviation back to the spec-conformant shape, that’s a separate refactor-prompt gap worth a parallel investigation. R50’s refactor phase was 2:52 wall-clock; if it did walk the spec, it apparently didn’t escalate the mismatch.

Mapping to the Research

Predicted (pbc-research)	Observed (pbc-5051)
Wide pair-range	green-phase +2.79 min on a 5–8 min baseline
Each run individually within in-control band	yes — both within the typical green-phase range for this scenario
Both runs pass the test	yes — both committed `run-rgr <scenario>` cleanly
Two work-trees differ	yes — different file families patched (TestStep* vs Row*)

The signature looks like pbc-4445 case 1 (“different files patched”) at the artifact level, but the root cause is not spec ambiguity — it’s a prompt-file mismatch (identify.md references uml-interaction.md which doesn’t exist; the actual spec lives in uml-interaction-main.md). The earlier same-scenario pairs converged on the Row* path; they didn’t surface this latent prompt gap because the model happened to read in the right order. This study makes the gap visible: the spec answer was always documented; the prompt just doesn’t lead claude to the file that contains it.

Findings by Variable

Each subsection records this run’s findings about one Wheeler variable. Read the same heading across the run sequence to see how our understanding of that variable evolved.

green time pair range

The widest pair-range on the gsheet for the Rebuild50–51 pair: 2.79 min (166,869 ms), with the two branches patching different file families (TestStepIssueDetector / TestStepIssueTypes vs new RowIssueDetector / RowIssueTypes). The vagueness was traced not to spec ambiguity but to a prompt-file-reference gap: identify.md cites ${umlDir}/uml-interaction.md, which does not exist on disk; the answer (RowIssueDetector.validateCellListWorkspace, by name, with a code template) lives in uml-interaction-main.md, which neither run opened. With no spec to anchor on, R50 fell back to extending the existing TestStepIssueDetector it could see while R51 inferred the row-owns-this convention from ValidateActionImpl’s dispatch branches — so the pair diverged on which file to patch and the range went wide.

green time pair range moving range

No finding this run.

green time

The longest-affected scenario in the series: green phase was 473,823 ms (R50) vs 306,954 ms (R51) on a 5–8 min baseline. R50 needed a second mvn verify cycle after its first edits failed compile/test; R51 passed first try. Neither run hit a contradiction or timeout — both committed cleanly.

green time moving range

No finding this run.

scale & green tokens

The two runs executed at different times (R50 May 25 01:21 EDT, R51 May 25 11:46 EDT), so server load and decode-rate / time-of-day noise is a confound on the raw wall-clock delta. That confound was only separated out later by token-scaling and concurrent same-time pairing; here the prompt-file-reference cause is primary and the time-of-day effect secondary.

warm-up position

No finding this run.

Open Questions From This Case

Did TestStepIssueDetector gain new surface area between Rebuild45 (the last “same-files” pair on this scenario) and Rebuild50 that made it a more plausible target? A git log --follow on the file across that window would answer it; if yes, the original spec ambiguity was hidden by lack of alternatives, and recent code growth surfaced it.
After the test-tightening fix lands (whatever issue number that becomes), re-running this scenario 2–3 times on the same branch will show whether the pair-range distribution narrows below the assignable-cause threshold. If it does not, the next assignable cause becomes visible (likely model-of-the-day variance on which detector class to invent).
The +77 s refactor delta (R51 actually longer refactor at 4:09 vs R50’s 2:52) is in the opposite direction of the green delta — worth a parallel study. R51’s refactor likely spent extra time because it had new files to inspect against UML specs (RowIssueDetector / RowIssueTypes); R50’s edits were in existing classes the refactor walker had cached models for. Different cause site, different prompt — separate investigation.