#1069 submit_workflow ignores queue= when the process already has a JumperClient: platform workflows meant for highway_internal run on highway_default; tenant onboarding fails at create_rate_limits
Description
EditEARS SPEC:
- When submit_workflow is called with queue=Q, the engine shall create the workflow_run with queue_name Q and spawn tools.workflow.execute on queue Q, whatever queue the process's shared JumperClient was created for.
- When get_jumper_client(queue_name=Q) is called and the shared client serves a different queue, the engine shall return a client for Q, not the shared one.
- When the platform API creates a tenant, the onboarding workflow shall run on a highway_internal worker and complete, including create_rate_limits against TokenGate.
FOUND 2026-10-25 06:07Z (UNATTENDED SESSION) by the #1036 live check: onboarding run 3bf19834 (zz1036_10250607_onboarding) failed at create_rate_limits: TokenGateUnavailableError: TokenGate key file /run/highway/tokengate_admin.key is not readable (FileNotFoundError). It ran on highway-worker-1 (queue highway_default), which by design does not mount tokengate_admin.key; highway-api and the internal workers do. Cause: engine/jumper_client.py get_jumper_client() is a process singleton and uses queue_name only on its first call, so platform_tenants_api.py:163, platform_tenant_lifecycle_api.py:146 and :251, and platform_bootstrap_service.py:192 (all queue=INTERNAL_QUEUE) enqueue on the API's default queue. Impact: every tenant created through the API since rate limits moved to TokenGate stops at create_rate_limits (no TokenGate assignment, no apps, no welcome email), and platform-trusted internal-tool workflows run on tenant-facing workers.
SYNTHESIS: per-queue client cache (a client per queue name), not one singleton for all queues. Rejected alternative: pass a queue argument down to spawn_task, because jumper's spawn_task and create_workflow_run read self.queue_name, so it would need changes in the vendored jumper and every caller.
Comments
Loading comments...
Context
Loading context...
Audit History
View AllLoading audit history...