Recovery without a correctness gain.

Our previous negated-closure comparison left an intended routing correction unmeasured because the updated call failed. We next improved call handling while keeping the routing prompt, evidence validator and rules baseline unchanged. Then we tested a fresh set of 40 fictional support requests.

One first attempt returned HTTP 503. The bounded retry returned a valid technical decision, restoring valid output from 39/40 to 40/40. But that route did not match the frozen review label. Exact matches stayed 38/40. The rules baseline matched 29/40; the guarded model view matched 33/40.

A response can be available, schema-valid and supported by a source-exact quote, yet still propose the wrong route under the reference. The recovered case illustrates that distinction. It does not establish that retries generally help or hurt routing correctness.

All 15 ordinary technical, access and billing requests with negated closure wording matched their frozen labels. Active closure, ambiguous closure, security/injection and vague controls each matched 5/5. These fresh-set results do not establish a new negation-policy gain: the prompt was unchanged and there was no independent old-adapter control.

Bound retries and preserve failures.

The optional createReliableGeminiAdapter wraps the historical adapter through an explicit entry point. It permits at most two attempts per classification, 30 seconds per attempt and 65 seconds overall including retry wait. Only timeouts, HTTP 503 and HTTP 504 permit one retry. Default jittered backoff is 4–4.8 seconds.

Supported Retry-After integer seconds or IMF-fixdate advice may extend the wait up to eight seconds. Unsupported or excessive advice defers retry and stops the batch. Other provider/body retry hints and clock-skew correction are not implemented. Invalid or blocked output, generic transport errors and successful-but-wrong decisions are not retried. Authentication, quota and specified request errors stop the batch, as do three consecutive unrecovered transient classifications.

The runner preserves each attempt, error, wait and known usage instead of replacing the first failure with the final success. Forty cases and two attempts per case cap the run at 80 API attempts. We used 41. No business actions were executed. A timeout does not guarantee that remote processing stopped; this is not an exactly-once action executor.

The historical adapter remains unchanged; the wrapper does not silently replace existing consumers. Software tests establish the handling mechanics, not a general live reliability improvement.

Fresh cases, separately reviewed by AI.

One AI agent authored 40 fictional English cases from the written routing policy, five in each of eight groups. Cases were shuffled before IDs were assigned. A separate agent in a fresh context saved initial labels before seeing either the author's proposals or assigned groups. Initial agreement was 40/40, with no disagreements, replacements or unresolved labels. Twenty-three labels require review and 17 permit a route proposal.

This was AI review, not human adjudication. Blinding the reviewer to groups reduces one cue; it does not remove shared training or policy-interpretation errors. One disputed medical-context label below shows why agreement alone cannot establish ground truth.

The implementation was frozen before the implementer read the fresh cases. Hashes bind inputs, author proposals, initial reviewer labels, final labels, policy and implementation. No exact normalized body was reused from seven older sets. That check does not establish conceptual novelty or training-data exclusion.

The run used Gemini 3.5 Flash Lite, the unchanged v7 routing prompt and quoted-v1 evidence contract, with 6.5-second spacing between classifications. Only ID, subject and body entered provider requests. Labels, groups and rationales did not. Guarded final decisions were derived without extra model calls.

Keep every case in the denominator.

Every view retains all 40 cases; failed outputs receive no correctness credit.
MeasureRulesFirst attemptFinal boundedGuarded final
Exact matches / 4029383833
Valid decisions / 4040394040
Required review caught / 2319212123
Unnecessary valid reviews7007
Correct route proposals / proposals10/1417/1817/1910/10

Operational review fallback after a failed call earns no classification credit. First-attempt review recall above counts valid model decisions only. The final guarded view caught all 23 review labels but imposed seven unnecessary reviews; its narrower proposal coverage is part of the result.

The first-attempt and final views come from the same run. First attempts already used the new 30-second deadline. They are not an independent old-15-second control. No first attempt exceeded 15 seconds, so this run showed no observed benefit from extending that deadline. It cannot isolate a causal deadline effect or establish stable failure rates.

Both review misses, including a disputed label.

v8-014: recovered, valid and wrong under the reference.

The fictional request mixed personal debt repayment notes, overdue loans and hardship arrangements into a business invoice export, then asked for help separating them. The frozen label requires review for sensitive financial context.

The first attempt returned HTTP 503 after 614 ms. After a scheduled 4,540 ms wait, the retry returned valid technical output in 647 ms, quoting “Invoice export mixes personal debt repayment notes into the business ledger”. Total classification latency was 5,801 ms; the successful retry reported 962 tokens. The exact quote identified the mixture, but the route still missed the review label. Availability recovered; correctness did not.

v8-016: the second miss also exposes label uncertainty.

The request described a journal tracking a fictional chronic illness, with symptoms and medication changes, then reported entries disappearing after sync. The model returned technical and quoted “entries vanished after sync”. The frozen AI-reviewed label requires review for medical context.

Explicit fictional framing makes the sensitivity interpretation debatable. The author and reviewer agreed, but may have shared an overly conservative interpretation. We retained the frozen label and primary scores, without favorable post-result relabelling or an alternative headline score. Human adjudication is needed before calling this a definitive real-workflow failure.

The guard reviewed both cases for lexical reasons: multiple billing/technical routes on v8-014 and no supported rules route on v8-016. That does not establish semantic sensitive-data detection. The rules baseline made 11 mismatches overall—four missed review labels and seven unnecessary reviews. The guarded view retained those seven unnecessary reviews. Every case-level mismatch and all eight group results are in the linked report.

There were no final invalid or unavailable decisions and no unattempted cases. The completed workflow intentionally retains a failure conclusion because its initial HTTP 503 remains recorded, even though the retry produced valid output. We did not rerun the experiment to erase that failure.

What this evaluation cannot establish.

Known provider-reported tokens across all attempt logs totaled 38,099: 37,137 on first attempts and 962 on the retry. Usage on the initial 503 is unknown, not zero; these figures are not a complete billing statement. Median first-attempt latency was 2,914 ms; median final classification latency including retry wait was 3,207 ms. Both nearest-rank p95 values were 11,115 ms. Deliberate spacing is excluded.

One fixed shuffled order, one live run and one recovered failure do not establish a reliability rate or latency guarantee. There was no randomized legacy control, repeated representative traffic or human routing adjudication. Forty policy-directed synthetic English cases, five per group, cannot establish customer performance, production readiness or comprehensive injection resistance. Earlier sets differ and cannot form an accuracy progress curve.

AI author and reviewer can share model, training and policy blind spots. Initial agreement is not evidence of independent human judgment. Source-exact quotes do not establish evidence relevance or completeness; there was no independent evidence-support adjudication. The now-seen set should not become a tuning target followed by claims of fresh evaluation.

More attempts can increase latency and cost. The runner checkpoints after each classification rather than each attempt; interruption can leave up to two calls outside the last saved checkpoint. A partial record must not be treated as complete or safely rerun. Sanitized diagnostics omit full provider error payloads and rejected raw responses. No new credentials, billing changes, quota evasion or business integrations were used.

Inspect the frozen record.

All 71 implementation and provenance tests passed locally without skips and on GitHub Node 22/24 before live calls. Offline verification reconciled scores, exact quotes, baseline and guarded decisions, attempts, known/unknown usage and provenance hashes without further model calls. These checks establish mechanics and traceability, not semantic improvement.

Publication note: this article was AI-drafted and checked against the frozen experiment artifacts. Cases and reference labels were AI-authored and separately AI-reviewed as described above; no human adjudication is claimed. No new model calls were made to prepare this article.

Next experiment: sensitive-context scope, a stopped evaluation and a separate continuation preserving the original failures.

← Back to the journal