#163 Activity workers missing DataShard logging - no audit trail for long-running tasks

closed high Created 2025-11-29 04:35 · Updated 2025-11-29 05:11

Description

Edit
## Problem Activity workers (highway-activity-worker@*.service) execute long-running tasks like Kafka producers/consumers but have NO logging to DataShard. This means: - No audit trail for activity execution - No way to trace what happened inside activities - Cannot correlate activity logs with workflow execution - Regulatory compliance gap for bank-grade requirements ## Current State - Workflow workers have DataShard logging via `tools.datashard.log_task_execution` - Activity workers create DurableContext but `event_logger=None` (see activity_worker.py:301) - Activity results are only stored in activity_queue.result column - Logs only available via systemd journal (`journalctl -u highway-activity-worker@*`) ## Expected Behavior - Activity execution should log start/complete/fail events to DataShard - Each activity should have immutable audit log entries - The long running programs logs (for example logging.info/debug/error) must be capture into datashard and can be found. - Logs should be queryable via DataShard time-travel queries - Activity logs should be linkable to parent workflow run ## Files to Modify - engine/services/activity_worker.py - Add DataShard logging in _execute_activity() - Possibly create activity-specific DataShard log table ## Context Discovered during Kafka ETL pipeline implementation (Issue #162). User asked 'how can I find producer and consumer full logs?' - answer was 'only in systemd journal' which is unacceptable for production. ## note: - restart server with `make restart`, don't do it by pkill - datashard's source code is in ../datashard/ - think about scalability, if audit log will be massive, we may need to partition it? what about other data - example program is kafka etl pipeline which we produced in #162

Comments

Loading comments...

Context

Loading context...

Audit History

View All
Loading audit history...