← Back
agent-loop blocker seeded

Empty search results make an agent keep renaming the same missing record

June 11, 2026

Symptom

A support agent asked to attach the March renewal addendum for account `acme-eu` spends the whole run searching documents instead of answering. The trace shows 31 successful `policy_doc_search` calls, all HTTP 200 and all returning `[]`, while the query text drifts from `March renewal addendum` to `ACME EU renewal rider` to `contract amendment Q1`.

Root Cause

The search tool encoded an empty hit list as a normal successful observation with no terminal meaning. The planner policy treated `[]` as weak evidence that the last query wording was poor, not as evidence that the indexed corpus lacked the requested document, so it repeatedly generated broader aliases against the same Meilisearch index. Because the loop state tracked only transport failures and not consecutive semantic misses for `(tool, account_id, date_range)`, every 200 response reset progress instead of moving the agent to a user-facing not-found branch.

Diagnosis Steps

  1. Open the run trace and count `policy_doc_search` calls where `status=200`, `hits=[]`, and `account_id=acme-eu`.
  2. Compare the query strings and verify they are paraphrases over the same account and March 2026 date range, not independent retrieval targets.
  3. Run `curl 'http://localhost:7700/indexes/policy_docs/search' -H 'Content-Type: application/json' --data '{"q":"March renewal addendum","filter":"account_id = acme-eu"}'` and confirm Meilisearch returns `hits: []`.
  4. Check the ingestion log for `policy_docs` and verify `acme-eu-renewal-addendum-2026-03.pdf` never appears in the indexed document IDs.
  5. Inspect the agent state reducer and confirm only exceptions increment `tool_failures`, while empty successful result sets have no counter or stop condition.

Fix

Changed `policy_doc_search` to return an explicit miss object after a valid empty result: `{"status":"not_found","query":"...","scope":{"account_id":"acme-eu","month":"2026-03"}}` instead of a bare array. The LangGraph node now increments `semantic_miss_count` for the same tool and scope, stops after three misses, and routes to a not-found response that names the searched account, month, and index. A nightly ingestion check also alerts when Salesforce attachments referenced by `renewal_addendum_id` are absent from the Meilisearch `policy_docs` index.

Prevention

Define empty-but-valid tool results as first-class outcomes, track repeated misses by search scope, and test that a missing document produces a not-found answer within a fixed call budget.

Stack

OpenAI Responses API LangGraph 0.2.45 Python 3.12 Meilisearch 1.9.0 FastAPI 0.115.0

Tags

agent-loop empty-results search tooling

Date

June 11, 2026

31 calls, every one scoped to account_id=acme-eu, drifted from March renewal addendum to ACME EU renewal rider to contract amendment Q1 without ever leaving the March 2026 attachment slot.

The first repeated miss in the run log was:

tool=policy_doc_search status=200 account_id=acme-eu q="March renewal addendum" hits=[]

From the agent’s point of view, an empty array was still an observation. It did not carry the meaning “the searched scope has been exhausted.” The next plan step therefore chose a familiar retrieval move: rename the thing and try again. By call 17, the agent had searched for renewal rider, signed addendum, March contract amendment, Q1 commercial terms, and ACME EU uplift letter; all were aimed at the same account and the same March 2026 attachment slot.

The missing document was not in Meilisearch. Salesforce had renewal_addendum_id=0688c00000Jx91QAAR, but the ingestion job skipped that attachment after a MIME-type filter rejected application/pdf; charset=binary. Once the index returned zero hits for the scoped search, more synonyms could not change the answer. The loop existed because the agent state had counters for failed calls but none for successful misses.

The repair made “nothing found” a typed outcome. Three empty searches against the same (tool, account_id, month) scope now route to a not-found branch, and the final message says the agent searched policy_docs for acme-eu March 2026 but the addendum is absent from the index. The ingestion monitor fixed the upstream gap; the agent fix prevents the next missing file from turning into 31 polite searches.