Keep both denominators.
We changed how our experimental support-triage adapter selects evidence. The new fact-anchor pipeline, v14, can represent source spans that the bounded v12 candidate menu misses. In a fresh paired evaluation, it corrected one such case. It also finished with one unavailable response after two HTTP 503 failures.
There was no observed overall gain. Joint success—correct routing, AI-rated decision support and coverage of a sufficient reference bundle—was 19/20 for both adapters. On the 19 cases where both produced valid responses, the joint score improved from 18/19 to 19/19. That conditional comparison isolates the available pairs; it does not erase the failed classification.
| Measure | v12 bounded selector | v14 fact anchors | Rules |
|---|---|---|---|
| Valid final responses / all cases | 20/20 | 19/20 | 20/20 |
| Correct routing / all cases | 20/20 | 19/20 | 9/20 |
| AI-rated evidence support / all cases | 19/20 | 19/20 | Not graded |
| Joint success / all cases | 19/20 | 19/20 | Not graded |
| Joint success / common-valid pairs | 18/19 | 19/19 | Not graded |
Both adapters used Gemini 3.5 Flash Lite and the unchanged routing policy. The 20 fictional English cases covered six multi-intent cases, six deliberately near-limit spans, four meaningful-subject cases and four protective controls. The baseline matched 9/20 labels, caught 7/14 required reviews and added four unnecessary reviews. Its 11 misses are preserved individually. Combining the model with the unchanged guard yielded 16/20 correct routes for v12 and 15/20 for v14, with four unnecessary reviews each.
Select facts, then extract.
The public contract stays narrow: a category and one unchanged quote of 4–120 JavaScript UTF-16 code units from a single original subject or body. A valid substring is not proof of sufficient evidence.
v12 offers a sampled menu of candidate spans, capped at 64 choices. A sufficient quote can be absent even when the original request contains it. v14 instead exposes every original word/punctuation token endpoint. The model selects one sufficient fact span, or exactly two nonoverlapping spans for multiple substantive intents. The adapter extracts the smallest contiguous source span enclosing those facts, including any neutral text between them.
Validation checks the reason/category relationship, fact count, token identity and order, field boundaries, overlap and quote length. A narrow English trailing-determiner check rejects some obvious cut endings. Invalid anchors and self-reported unavailable evidence fail closed; there is no quote repair or semantic retry.
Those are structural checks. Two selected spans can still describe the same intent, omit a qualifier or support the wrong decision. Meaningful subjects remain available, so the model must distinguish decisive facts from generic headings. Token endpoints cover same-field word/punctuation-aligned spans, not arbitrary slices within a token. Cross-field or discontiguous evidence cannot be stitched together.
One missing witness restored.
Case v14-013 describes invoice duplication and a separate login failure, separated by neutral prose. Under the frozen reference, both concrete symptoms are needed to establish multiple intents. The independently observed sufficient quote is 116 units long.
v12 returned review with this source-exact quote:
Here is what I saw: Invoices duplicate.
The route matched the label, but the quote contained only invoice duplication. The blind evidence reviewer judged it unsupported for the returned review decision and incomplete against the sufficient bundle. The pre-run mechanical audit found no v12 candidate containing the reviewed witness; that containment test alone does not prove the absence of every conceivable sufficient alternative.
v14 selected both facts and returned:
Invoices duplicate. My note is on plain paper beside a green pen and a mug near the office window. Also, login fails
The reviewer judged this quote supportive and reference-complete. The new representation also restores all six historical boundary witnesses in offline regression checks; historical live scores were not revised or retested. In this fresh set, all 20 reviewed witnesses are token-aligned and representable by v14, while 19 are contained in v12 candidates. Representability is a capability of the source map, not a guarantee of correct selection.
Every failed attempt remains.
All 40 planned classifications completed in 46 provider attempts, within the cap of 80. Six first attempts failed; five calls recovered on their single permitted retry. One call failed twice, giving seven failed attempts overall: four HTTP 503 responses and three timeouts.
| Case / adapter | First failure | Final outcome |
|---|---|---|
| v14-016 / v14 | HTTP 503 | Second HTTP 503; unavailable |
| v14-004 / v12 | HTTP 503 | Recovered; correct review |
| v14-006 / v14 | Timeout | Recovered; correct billing |
| v14-011 / v12 | Timeout | Recovered; correct access |
| v14-002 / v12 | HTTP 503 | Recovered; correct review |
| v14-002 / v14 | Timeout | Recovered; correct review |
The unavailable v14-016 response earns no routing or evidence credit. It is not a valid wrong route, and its review fallback is not a successful classification. Neither valid model arm had an observed routing or review miss; all 39 valid responses matched their labels. First-attempt routing was 17/20 for each adapter.
Transport limits were 30 seconds per attempt, 65 seconds per classification and at most two attempts for the inherited transient conditions, with 30 seconds between classifications. No quota workaround, billing change, project/key change, additional probe or result replacement was used. The original workflow retains a failure conclusion because its exit condition records first/final call errors even though coverage is complete. The raw checkpoint has no unattempted or uncertain in-flight call.
Recorded total usage is 96,746 tokens: 44,523 for v12 and 52,223 for v14. Seven failed attempts lack usage metadata; unknown is not zero. These arm totals also cover different numbers of valid responses, so they are not a matched cost estimate. Larger token maps and schemas are a practical overhead to measure, not a free improvement.
Exact can still be unfinished.
Two valid quotes retain awkward endings. v12's v14-016 quote ends with an extra “The”:
Payments fail. The desktop has the standard wallpaper, and the room has a shelf near the window. Also, maps freeze. The
Both substantive symptoms are complete before that trailing fragment. The blind reviewer therefore accepted support and bundle coverage, while explicitly recording the rough ending.
v14's v14-003 quote ends with “are”:
My actual chemotherapy results are
The reviewer judged the actual personal chemotherapy-results disclosure independently sufficient for sensitive-context review. It did not require the optional export placement and column symptom. That semantic judgment does not make the sentence finished. Fact anchoring did not prevent every trailing fragment or produce polished evidence. No invalid anchor or length rejection was observed among the returned decisions in this run; the unavailable call never yielded a decision to inspect.
What blind AI review means.
The implementation, harness, policy and rubric were frozen before fresh authorship. A separate fresh-context AI author created cases. Another AI reviewer independently saved routing labels, sufficient fact bundles and shortest-observed quotes before seeing author labels, groups or examples. Initial routing agreement was 20/20. Pre-call reconciliation preserved four criteria/span refinements, including a shorter independently sufficient classifier-directed command. Agreement was not treated as human adjudication.
All six boundary witnesses were independently observed at 114–116 units. Their first drafts used repetitive framing. A permitted second prospective round changed only neutral prefixes/suffixes and placement, keeping the core symptoms and witnesses unchanged. Original drafts, temporary rejected whitespace padding, initial judgments and every revision remain in the record. No provider outcome informed these changes. The final cases still share a deliberately constructed symptom/neutral-gap pattern.
A third fresh-context AI reviewer received 39 shuffled valid final responses with pseudonym IDs, full requests, expected decisions and sufficient fact criteria. It did not receive arm identities, original case IDs, groups, exemplar/shortest quotes, selected anchors/reasons, code, prompts, run order or aggregate scores. Grades were saved and hashed before unblinding.
The written protocol itself disclosed the conceptual anchor approach. These agents were not blind to every algorithm concept. The primary implementer knew historical failures and supplied disclosed design feedback after freezing the source. Repeated requests can reveal pairing, and output style can suggest an arm. Shared training, policy assumptions and AI blind spots can correlate errors. The evidence reviewer raised no material criteria ambiguity here; that is not proof that the references are complete or correct. No human label or evidence adjudication has occurred.
The boundaries of this result.
This is one small, targeted English synthetic run. Source representation, schema, prompt and validation changed together; there is no isolated causal attribution, significance claim or stable failure-rate estimate. A novelty audit found no exact body duplicates against 442 prior requests, but one reused heading and a highest word-set Jaccard similarity of 0.75. That does not establish semantic novelty or training exclusion.
Shortest-observed spans are search judgments, not mathematical minima. The English cut heuristic can reject legitimate wording. Live bodies were at most 500 units; full 2,000-unit inputs, provider schema complexity at that limit and multilingual behavior remain unmeasured. Optional context coverage is reported separately from the sufficient-trigger score.
All 140 software tests passed on both Node 22 and 24, and the saved-result audit reproduces counts, source extraction, frozen files and review hashes offline. Those checks establish software behavior and provenance, not human validity or production readiness. The adapter remains explicit and experimental; no default adapter was promoted and no business action was executed. Publication used the preserved records and made no new model calls.
Inspect the record.
- Full findings, every failure, group scores and limitations
- Original paired outputs and attempt logs, safe diagnostics and usage accounting, and all-case/common-valid evidence grades
- Pre-run witness containment audit and novelty checks
- Original-to-final reference reconciliation and prospective framing revisions
- Blinded items, sealed grades, review process and unblinding map
- Frozen pre-authoring protocol, offline verifier and original live workflow with its failures
Publication note: AI-drafted and checked against the preserved evaluation. Synthetic cases and AI grades should be independently reviewed by people before operational conclusions.