#833 CATASTROPHIC: /simulate re-executed REAL tool side effects + crashed on incomplete mock client
Description
EditUser report: opening the Simulate/Replay tab on a real workflow produced a 500 with 'ReplayJumperClient object has no attribute set_checkpoint', AND the traceback showed a live shell task (rm -f /data/demo/...) actually being executed during simulation — stopped only by the shell security validator.
TWO defects:
1. SAFETY (catastrophic): simulate re-executed the workflow with the NORMAL InlineExecutor + TaskOperator, which calls tool_func(ctx, *args) directly (task.py:179). ReplayContext was designed to mock side effects but the task path never routed through ctx.step() — so shell/http/email/etc. ran FOR REAL during a 'dry run'. The dashboard even advertised 'no side effects'. An http.post/email/rm simulate would fire real effects.
2. CRASH: on any task failure the retry path calls jumper_client.set_checkpoint, absent on the stub ReplayJumperClient → AttributeError inside the failure handler → 500, masking the real error.
ROOT: plain-task results are stored as VARIABLES, not per-task checkpoints, so replay could only reproduce them by re-executing — the 'mocked side effects' premise was false for every tool-executing task.
FIX (shipped): simulation is now side-effect-free — TaskOperator returns each task's RECORDED result from the step_succeeded audit events (ReplayContext.get_recorded_result) instead of invoking the tool, guarded by ctx.is_replay (normal execution byte-identical). ReplayJumperClient gets a __getattr__ no-op backstop so no operator can AttributeError the simulation again. Dashboard copy corrected 're-executes' → 'replays the recorded run'. Tests: tests/integration/test_simulate_side_effects.py (deterministic across calls = tool not re-executed; failed-run simulate no longer 500s).
DEFERRED (separate ticket if wanted): 'true what-if re-execution' (resume_from with fresh logic) would require mocking side-effecting tools while re-running pure logic + per-task result checkpoints — a real design effort. Current simulate is an honest, safe REPLAY of the recorded run + checkpoint schema-mismatch detection.
MY MISS: I surfaced Simulate prominently in the #824 consolidation (ActionBar link + 'no side effects' hint) and claimed in the plan/memory it was a safe dry-run WITHOUT ever running an actual simulation — E2E'd only the buttons. Lesson: exercise the feature's actual backend path, not just the UI that links to it.
Comments
Loading comments...
Context
Loading context...
Audit History
View AllLoading audit history...