#671 Add OrphanDetectionService for stuck workflows
Description
EditNeed a background service that periodically scans for orphaned workflows where:
- workflow_run.status IN ('running', 'sleeping')
- BUT queue entry is completed/failed/cancelled/missing
These orphaned workflows will never complete without manual intervention.
The service should:
1. Run every 5 minutes
2. Find orphaned workflows older than 5 minutes (to avoid false positives during normal processing)
3. Mark them as 'failed' with appropriate error message
4. Log the cleanup for audit trail
SQL query for detection:
SELECT wr.workflow_run_id, wr.workflow_name, wr.status, r.state
FROM workflow_run wr
LEFT JOIN absurd.r_highway_default r ON r.run_id = wr.absurd_run_id
WHERE wr.status IN ('running', 'sleeping')
AND (r.state IS NULL OR r.state IN ('completed', 'failed', 'cancelled'))
AND NOW() - wr.created_at > INTERVAL '5 minutes';
Comments
Loading comments...
Context
Loading context...
Audit History
View AllLoading audit history...