#1034 Activity worker process exits when _fail_activity raises on the bulkhead-submit failure path

open medium engine resilience Created 2026-10-20 07:05 · Updated 2026-10-20 07:05

Description

Edit
Found during #906. engine/services/activity_worker/worker.py _submit_activities: if bulkhead.execute raises, _fail_activity is called; _fail_activity re-raises any DB error, which leaves _run_loop, start() logs 'crashed' and re-raises, so the activity worker process stops (compose restarts it; in-flight activities of that process lose their heartbeats and are recovered only after the 60 s stale threshold). Same in both former copies. Fix direction: catch and log on that path (the activity stays running and is reclaimed as stale), red-first unit test with a failing fake DB.

Comments

Loading comments...

Context

Loading context...

Audit History

View All
Loading audit history...