>_
.issue.db
/highway
Dashboard
Issues
Memory
Lessons
Audit Log
New Issue
Edit Issue #825
Update issue details
Title *
Description
POST /workflows/<id>/cancel only calls jumper cancel_task on the wrapper (tools.workflow.execute) run. A worker claims that wrapper within ~1-3s of submission and holds the claim for the workflow's ENTIRE life, so cancel_task refuses ('Could not cancel task run') and the endpoint 409s. There is NO fallback: nothing marks the run for cooperative cancellation, so the workflow always runs to completion. Evidence (2026-07-19, prod): - 15 consecutive cancel attempts over ~60s (~6 step boundaries of a 30x sleep-10 chain, run 83464b28) -> all HTTP 409, run continued. - Runs 6d2f34ac + 9efa7727: dashboard Cancel click -> 409 -> both ran to completion. - Cancel only ever succeeds pre-claim (run b529db84: cancelled 2s after creation, wrapper unclaimed). - Pause is COOPERATIVE and works while claimed: endpoint flips workflow_run.status='paused'; engine.orchestrator.execution_helpers.check_workflow_paused (called per DSL task, orchestrator/core.py:155) re-queues tasks. No equivalent exists for cancel. Proposed fix (mirror pause): 1. api cancel_workflow: when jumper cancel_task fails on a claimed run, set workflow_run status='cancelling' (or cancel_requested_at) and return 202 instead of 409. 2. engine: extend the per-task boundary check to abort on 'cancelling' -> finalize run as 'cancelled' (in-flight DSL task completes, nothing new starts; recovery/on_failure must NOT fire). 3. Sweeper reconciliation for orphaned 'running' rows whose wrapper run is gone (see run 812a91a8: 'running' 50+ min for a ~2min chain; cancel 409s 'may have already completed'). Dashboard note: highway-dashboard e2e/action-bar.mjs asserts in-flight cancel surfaces the 409 conflict; flip that assertion to real cancellation when this lands. Historical 'cancel works at step boundaries' belief was a contention fluke (wrapper unclaimed for 60s+ under load).
Priority
Low
Medium
High
Critical
Status
Open
In Progress
Closed
Won't Do
Due Date (YYYY-MM-DD)
Tags (comma separated)
Related Issues (IDs)
Enter IDs of issues related to this one. They will be linked as 'related'.
Update Issue
Cancel