#783 Gmail auto-responder dedup is non-deterministic (LLM-judged, sender-per-day) -> 26-min delay + skipped/duplicate replies

closed high agents mcp reliability Created 2026-06-29 20:19 · Updated 2026-06-29 22:54

Description

Edit
Watch auto-responder (_watch_goal in highway-mcp/server.py) enforces reply-idempotency via an LLM INSTRUCTION ('search Sent; if already replied to this sender today, skip') instead of deterministic state. Evidence (tenant demo, 2026-06-29): incoming 'Highway support' from rodmena@me.com at 19:38:55; ~13 poll runs over 26 min each independently judged the dedup; most SKIPPED, one finally answered at 20:05:06. Smoking gun: run 23fc7b48 (20:02) ANSWERED while run 94ae0d17 (20:04) SKIPPED on IDENTICAL inbox state -> non-deterministic. Three defects: (1) idempotency by LLM judgment = nondeterministic; (2) dedup key is sender-per-day, too coarse (any mail to a sender marks the actual question 'handled'); (3) signal poisoned by a second rule (stale from:rodmena@me.com->joke push rule replied to same sender, which the dedup counted as 'already replied'). Also ~9/24 poll runs FAILED outright (separate reliability issue). Fix: deterministic per-MESSAGE idempotency - apply a Gmail label (e.g. highway-answered) and exclude it in search (needs gmail.modify scope), or deterministically check the thread for an existing reply to that message_id in code; scope dedup to the message, not the sender; remove/segregate conflicting rules. LESSON: never use an LLM to enforce idempotency/dedup - it must be deterministic state (processed-message-id set / label / thread check).

Comments

Loading comments...

Context

Loading context...

Audit History

View All
Loading audit history...