>_
.issue.db
/highway-workflow-engine
Dashboard
Issues
Memory
Lessons
Audit Log
New Issue
Edit Issue #721
Update issue details
Title *
Description
OBSERVED: Workers can crash with asyncio.exceptions.CancelledError during HTTP operations, causing all internal threads (TimeoutService, task processing loop, etc.) to die while the container remains 'running' and 'healthy'. Symptoms: - Container status: running/healthy - Zero log activity - Stuck workflows not cleaned up - TimeoutService not running Root cause: asyncio task group error during HTTP connection: asyncio.exceptions.CancelledError: Cancelled via cancel scope ... Impact: Workflows get stuck in 'running' state indefinitely because TimeoutService (which cleans up expired claims) is dead. Workaround: Restart workers with 'docker compose restart worker' Suggested fix: Add a watchdog that monitors thread activity and restarts worker if all threads are dead. Or improve asyncio error handling to prevent cascade failures.
Priority
Low
Medium
High
Critical
Status
Open
In Progress
Closed
Won't Do
Due Date (YYYY-MM-DD)
Tags (comma separated)
Related Issues (IDs)
Enter IDs of issues related to this one. They will be linked as 'related'.
Update Issue
Cancel