root
8fc88ffc14
Build & Push / Pipeline Tests (push) Failing after 59s
Build & Push / Build & Push Docker Image (push) Has been skipped
Test / Type Check (all packages) (push) Failing after 51s
Test / API Unit Tests (push) Has been skipped
Test / Homepage Unit Tests (push) Has been skipped
Test / Carplace Unit Tests (push) Has been skipped
Test / Admin Unit Tests (push) Has been skipped
Test / Dashboard Unit Tests (push) Has been skipped
Test / API Integration Tests (push) Has been skipped
57 lines
2.2 KiB
Markdown
57 lines
2.2 KiB
Markdown
# Runtime and ops surface (post Phases 1–2)
|
||
|
||
Design note for operators and engineers. Reflects the **current** intended production topology.
|
||
|
||
## Processes
|
||
|
||
| Process | Role | Notes |
|
||
|---------|------|--------|
|
||
| `api` | HTTP API | Do **not** set `ENABLE_EMBEDDED_JOBS=true` when running multiple API replicas |
|
||
| `api-worker` | Outbox + scheduled jobs | Uses DB leases + Redis leader lock for cron |
|
||
| Frontends | homepage, carplace, dashboard, admin | Unchanged modular apps |
|
||
| postgres | System of record | |
|
||
| redis | Rate limit, idempotency, pub/sub, locks | Required for replica-safe behavior |
|
||
| object storage | Optional S3/MinIO | Prefer over local disk for multi-replica files |
|
||
|
||
## HTTP ops endpoints
|
||
|
||
| Method | Path | Purpose |
|
||
|--------|------|---------|
|
||
| GET | `/health` | Liveness |
|
||
| GET | `/ready` | Readiness: database, redis, storage |
|
||
| GET | `/metrics` | Prometheus-style counters/gauges (latency, status, outbox pending/published) |
|
||
| GET | `/api/v1/openapi.json` | OpenAPI document |
|
||
| GET | `/docs` | Swagger UI (when enabled) |
|
||
|
||
Worker (optional):
|
||
|
||
| Env | Behavior |
|
||
|-----|----------|
|
||
| `WORKER_METRICS_PORT=<port>` | Worker listens for `GET /metrics` and `GET /health` |
|
||
| `0` / unset | No worker HTTP listener |
|
||
|
||
## Important environment knobs
|
||
|
||
Documented in `.env.example`:
|
||
|
||
- `ENABLE_EMBEDDED_JOBS` — default false in production examples
|
||
- `RATE_LIMIT_STORE` / `IDEMPOTENCY_STORE` — prefer `redis`
|
||
- `FILE_STORAGE_DRIVER` — `local` or `s3` (+ `S3_*`)
|
||
- `TRUSTED_FORWARD_HEADERS` — default false; see `docs/ops/proxy-trust.md`
|
||
- `ADMIN_FRESH_2FA_MAX_AGE_MS` — default `1800000` (30 minutes)
|
||
- `WORKER_METRICS_PORT` — optional
|
||
|
||
## Observability design
|
||
|
||
- Access logs: structured JSON via morgan in `app.ts`
|
||
- In-process series: `apps/api/src/lib/opsMetrics.ts`
|
||
- Durable outbox gauges: API `/metrics` also counts PENDING / PUBLISHED rows from PostgreSQL so scrapes remain useful when jobs run only on the worker
|
||
|
||
## Failure / release runbooks
|
||
|
||
- Chaos helper: `scripts/chaos/failure-injection.sh`
|
||
- Soak: `npm run test:soak` → `scripts/load/soak-probe.mjs`
|
||
- Canary / rollback: `docs/ops/canary-rollback.md`
|
||
- Key rotation: `docs/ops/key-rotation-drill.md`
|
||
- Backup smoke: `scripts/backup-restore-smoke-check.sh`
|