#625 Architectural: Don't hold DB transaction during long task execution

closed medium Created 2025-12-19 00:11 · Updated 2025-12-19 00:21

Description

Edit
**Current Behavior:** The inline_executor creates a SAVEPOINT before task execution (line 1560-1562) and releases it after storing results (line 1568). During task execution, the DB connection sits idle-in-transaction. **Problem:** Long-running tasks (>idle_in_transaction_session_timeout) cause PostgreSQL to kill the connection. Task completes but cannot store results. **Current Workaround:** Increased idle_in_transaction_session_timeout from 60s to 300s in docker-compose.yml. **Proper Architectural Fix:** 1. Execute tasks OUTSIDE of transaction scope 2. Only use transaction for checkpoint storage (quick operation) 3. Consider: task execution -> THEN open transaction -> store result -> commit **Affected Code:** - engine/interpreters/inline_executor.py:1559-1591 (execute_task_inline) - The SAVEPOINT pattern is for rollback on failure, but holds connection during execution **Risk if not fixed:** Tasks running >5 minutes will still hit the timeout. The workaround only extends the limit. **Alternative approaches:** 1. Periodic keepalive queries during task execution 2. Connection pool with health checks 3. Restructure to use optimistic concurrency instead of savepoints

Comments

Loading comments...

Context

Loading context...

Audit History

View All
Loading audit history...