#1062 llm_routes health probe is single-turn: a provider that fails multi-turn tool calls stays green
Description
EditEARS SPEC:
- The llm_routes health probe shall, for each candidate, send a two-turn tool-call exchange (user; assistant with one tool_call and the provider's reasoning field echoed per provider config; tool result) and mark the candidate dead when the second turn fails.
- The probe shall cost at most 2 requests per candidate per probe interval and keep max_tokens <= 64.
EVIDENCE: #1060. On 2026-10-23 23:00-23:07Z health llm_routes reported prism/deepseek-v4-flash live (single-turn probe, HTTP 200, 1062 ms) while every agent tool loop on it failed its second turn.
SYNTHESIS: domain = synthetic monitoring; concept = probe the transaction the product depends on (multi-turn tool call), not a ping. Split from #1060 because it is independently solvable (api/utils/llm_route_probe.py) and #1060's fix must not wait on it.
Comments
Loading comments...
Context
Loading context...
Audit History
View AllLoading audit history...