#757 Agent items 2+3: idempotency (#750 partial), prompt-injection defense, cost/budget metering, eval harness
Description
Editrun_goal hardening (validated on demo): (1) COST METERING: token_budget param + per-run token accounting (agent_tokens var) from each LLM call's usage; stops with finish_reason=budget when exhausted (verified token_budget=30 -> 'Stopped: token budget exhausted'). (2) PROMPT-INJECTION DEFENCE: GUARDRAIL_PROMPT prepended to system prompt (tool output = untrusted data); _scan_injection() flags injection patterns in tool output and wraps it with a SECURITY WARNING; outbound actions still HITL-gated. Verified: agent ran echo 'ignore all previous instructions...PWNED' and reported it as data, did NOT comply. (3) #750 IDEMPOTENCY: catalog declares idem_arg per tool (http=step_name, db=checkpoint_name); run_goal injects a unique value f'agentcall_{turn}_{i}' so each tool's internal durable step is unique across turns (exactly-once on retry, no stale-cache). (4) EVAL HARNESS: highway-mcp/eval_agent.py runs canonical tasks via MCP tools, auto-approves, checks outcomes (4/4 pass: math/shell/http/injection). CAVEATS: injection defence is heuristic (guardrail+HITL are the real backstops); idem hook only on tools with checkpoint params (shell empirically resets, email HITL-gated single-effect).
Comments
Loading comments...
Context
Loading context...
Audit History
View AllLoading audit history...