#844 circuit-breakers 503: Vault password for resilient_circuit_db never applied to the postgres role
Description
EditEARS SPEC:
- When GET /api/v1/circuit-breakers is called by an authorized caller, the Highway API shall return HTTP 200 with the circuit-breaker list.
- The Highway circuit-breaker database credential shall be verified against the live database at deploy time; if the stored credential cannot authenticate, then deployment shall fail loudly rather than leaving the endpoint dead.
- If the circuit-breaker database secret is absent, then the Highway API shall fail closed with an explicit configuration error and shall NOT fall back to a hardcoded default password (api/blueprints/v1/circuit_breakers.py:54 currently does `or "postgres"`).
- While the circuit-breaker database is unreachable or unauthenticated, the Highway API shall report that condition as a degraded check in GET /api/v1/health, not only on first use of GET /api/v1/circuit-breakers.
- The Highway platform shall hold exactly one source of truth for a given database credential; the main database and the circuit-breaker database shall not diverge silently.
EVIDENCE (reproduced 2026-08-03):
- GET /api/v1/circuit-breakers -> 503 DATABASE_UNAVAILABLE: connection to server at "172.25.0.2" port 5432 failed: FATAL: password authentication failed for user "postgres".
- 172.25.0.2 IS highway-postgres (the docker DB). The DB is up and healthy; only this one code path fails.
- Vault highway/shared/database -> {"value": "postgres"} (correct, matches the role)
- Vault highway/shared/circuit_breaker_database -> {"password": "<32-char random>"} (never applied to the role)
- docker/.env RC_DB_PASSWORD=postgres and POSTGRES_PASSWORD=postgres agree; only the Vault secret disagrees, and circuit_breakers.py reads Vault, not RC_DB_*.
- Reproduced byte-identical from inside highway-api: PGPASSWORD=<vault value> psql -h postgres -U postgres -d resilient_circuit_db -> same FATAL. PGPASSWORD=postgres -> connects.
- No code in either repo WRITES this secret, so it was seeded out of band and never reconciled.
- Related: #836 (services all connect as superuser postgres; a dedicated least-privilege rc role is the durable fix).
Comments
Loading comments...
Context
Loading context...
Audit History
View AllLoading audit history...