#517 [API/Memory] DataShard table unbounded memory loading
Description
Edit**File:** api/blueprints/v1/logs.py:90-119
**Problem:** `_read_table_to_pandas()` loads ENTIRE DataShard tables into memory on every request. With continuous workflow execution, these tables grow indefinitely. No pagination at DataShard level.
**Severity:** CRITICAL
**Fix:**
- Implement LRU cache with TTL for table snapshots
- Add predicate pushdown to DataShard queries
- Implement streaming/chunked reading
- Add memory limits per-query
**Impact:** Production systems with high workflow volume will exhaust memory.
Comments
Loading comments...
Context
Loading context...
Audit History
View AllLoading audit history...