# Mini Work Package: rq-engine JWT implementation (Phase 6 auth)
Status: Complete
Last Updated: 2026-01-12
Primary Areas: `wepppy/microservices/rq_engine/*`, `wepppy/weppcloud/utils/auth_tokens.py`, `tools/wctl2/commands/*`, `docs/dev-notes/auth-token.spec.md`, `docs/culvert-at-risk-integration/*`

## Objective
Implement JWT auth for rq-engine (FastAPI) and align it with session-based access so controllers can rely on `/rq-engine/api/*`. Provide a semi-permanent JWT flow for Culvert_web_app, a host-side `wctl` command to mint tokens, and a first-class revocation path.

## Scope
- rq-engine JWT validation and scope enforcement on culvert ingestion + job polling routes.
- Shared token contract and scopes using the existing WEPP_AUTH_JWT configuration.
- Flask `/rq/api/*` routes removed; JWT enforcement lives in rq-engine.
- `wctl` command to mint JWTs from the host with configurable TTL and claims.
- Minimal docs refresh so new agents can find the JWT surface area quickly.

## Non-goals
- rq/api migration tracked separately (see `docs/mini-work-packages/20260112_rq_api_migration.md`).
- OAuth or user-facing login changes.
- Reworking existing agent JWT flows (AGENT_JWT_*).
- Full webhook implementation beyond Phase 6 planning notes (tracked but not delivered here).

## Current JWT inventory (for new agents)
- HMAC JWT utilities: `wepppy/weppcloud/utils/auth_tokens.py` + stub.
  - Env config: `WEPP_AUTH_JWT_*` (see `docs/dev-notes/auth-token.spec.md`).
  - CLI issuer: `wepppy/weppcloud/_scripts/issue_auth_token.py`.
  - Used by `wepppy/weppcloud/routes/command_bar/command_bar.py` to mint Query Engine MCP tokens.
- Query Engine MCP auth: `wepppy/query_engine/app/mcp/auth.py` (separate env prefix `WEPP_MCP_JWT_*`; compose wires `WEPP_MCP_JWT_SECRET` from `WEPP_AUTH_JWT_SECRET`).
- Agent JWTs (Flask-JWT-Extended): `wepppy/weppcloud/utils/agent_auth.py`, `wepppy/weppcloud/routes/agent.py`, `wepppy/mcp/base.py` (AGENT_JWT_* env vars).
- Docker defaults: `docker/docker-compose.dev.yml`, `docker/docker-compose.prod.yml` define `WEPP_AUTH_JWT_SECRET` and propagate to query-engine.
- Tests: `tests/weppcloud/test_auth_tokens.py`, `tests/query_engine/test_mcp_auth.py`.
- No rq-engine auth yet: `wepppy/microservices/rq_engine/*` has no JWT enforcement.

## Token model (proposed)
- Issuer/validation: reuse `WEPP_AUTH_JWT_*` config and `auth_tokens.decode_token`.
- Audience: require `aud` includes `rq-engine` (or `wepp-services` if we choose a shared audience).
- Required claims: `sub`, `iat`, `exp`, `scope`, `aud` (plus `iss` if configured).
- Optional claims:
  - `runs`: list of run IDs for run-scoped RQ endpoints.
  - `culvert_batches`: list of batch UUIDs for culvert batch polling.
  - `session_id`: optional session identifier for anonymous session tokens.
  - `token_class`: `user|service|session` to simplify downstream policy checks.
  - `jti`: token ID (future revocation/rotation hooks).
- Scope set (initial):
  - `rq:status` - poll jobstatus/jobinfo.
  - `rq:enqueue` - enqueue jobs outside culvert flow (future).
  - `culvert:batch:submit` - POST `/culverts-wepp-batch/`.
  - `culvert:batch:retry` - POST `/culverts-wepp-batch/{batch}/retry/{point_id}`.
  - `culvert:batch:read` - read batch/job metadata.

### Token classes (new requirement)
- **User token**
  - Claims: `sub` (user id), `roles` (list), `groups` (list), optional `email`.
  - Authorization: rq-engine should resolve user roles/groups against the same logic as Flask (run ownership, Admin role, public run).
  - Avoid embedding full run lists (too long); use live checks against user/run metadata.
- **Service token**
  - Claims: `sub` (service id), `service_groups` (list), optional `env`/`purpose`.
  - Authorization: map `service_groups` to scope bundles (for example `culverts` grants culvert-specific scopes).
  - Use for Culvert_web_app and other service-to-service clients.
- **Session token (anonymous runs)**
  - Claims: `sub` (session id), `runid` (single run), `session_id`, `token_class="session"`.
  - Authorization: rq-engine allows run-scoped endpoints only when token matches `runid` and (optionally) the session is still active in Redis.
  - TTL: short-lived (15-60 minutes) and refreshable; `jti` required for revocation.

### Revocation and rotation (required)
- Require `jti` on issued tokens and validate against a denylist.
- Store revoked `jti` values in Redis DB 0 or 13 with TTL matching `exp`.
- Add rotation hooks (multiple secrets, or a primary + secondary secret list) so issued tokens can be rotated without downtime.
- Provide a minimal admin CLI to revoke a token by `jti` (or subject + issued-at window).

## Plan
### Phase 1 - Spec alignment + docs
- Extend `docs/dev-notes/auth-token.spec.md` with:
  - Token classes (user vs service), `service_groups` claim.
  - Revocation/rotation requirements (jti + denylist + TTL).
  - rq-engine scopes and audience guidance.
- Confirm the canonical error payloads follow `docs/schemas/rq-response-contract.md` (complete).

### Phase 2 - Shared rq-engine auth helpers
- Add `wepppy/microservices/rq_engine/auth.py`:
  - Parse Authorization header (Bearer token).
  - Validate token with `auth_tokens.decode_token` and enforce `aud`, `iss`, and scopes.
  - Resolve token class (user vs service) and normalize claims.
  - Helper functions: `require_scope`, `require_run_claim`, `require_culvert_batch_claim`.
  - Configurable auth mode: `RQ_ENGINE_AUTH_MODE=required|optional|disabled`.
- Keep helpers thin and avoid fallback wrappers that mask config errors.

### Phase 3 - Enforce JWT on rq-engine routes
- Apply dependencies in `wepppy/microservices/rq_engine/culvert_routes.py` and `job_routes.py`.
- Requirements:
  - `culverts-wepp-batch/`: `culvert:batch:submit`
  - `culverts-wepp-batch/{batch}/retry/{point_id}`: `culvert:batch:retry`
  - `canceljob`: secure with `rq:status` (or a new `rq:cancel` scope if desired).
- Explicitly leave `jobstatus/jobinfo` unsecured (read-only polling for agents).
- Batch/job claim checks:
  - Use job meta `culvert_batch_uuid` (already stored) to validate `culvert_batches` claim.
  - For run-scoped jobs, compare `job.meta["runid"]` to `runs` claim or allow if claim missing and auth mode is optional.
- Error responses must use rq response contract (`error`/`errors` keys).
 - Add `POST /rq-engine/api/canceljob/{job_id}` (parity with Flask, but secured).

### Phase 4 - Flask `/rq/api/*` removal (completed)
- Flask `/rq/api/*` endpoints were removed instead of retrofitting JWT helpers.
- All queue-triggering routes now live in rq-engine, which enforces JWT claims directly.
- Legacy `rq_auth.py` helpers were dropped along with the Flask routes.

**Completed now:**
- `POST /rq-engine/api/runs/<runid>/<config>/session-token` issues a session JWT and sets the Redis marker.
- Job dashboard cancel now uses `/rq-engine/api/canceljob` with a session token.
- rq-engine culvert ingestion/retry routes now require JWT scopes (`culvert:batch:submit`, `culvert:batch:retry`).
- Flask `/rq/api/*` routes removed; controllers now target rq-engine endpoints only.

### Phase 5 - wctl token minting command
- Add a Typer command in `tools/wctl2/commands/` (new `auth.py` or extend `python_tasks.py`):
  - `wctl issue-auth-token <subject> --scope ... --runs ... --audience ... --expires-in ... --claim key=value --json`
  - Runs `python -m wepppy.weppcloud._scripts.issue_auth_token` inside the `weppcloud` container so `WEPP_AUTH_JWT_*` config is available.
  - `--expires-in` supports semi-permanent culvert tokens (example: 90-180 days).
- Document in `wctl/README.md` with culvert-focused examples.

**Completed now:**
- `wctl issue-auth-token` added (wrapper for `wepppy/weppcloud/_scripts/issue_auth_token.py`).
- `wctl/README.md` updated with the auth token command example.

### Phase 6 - Revocation workflow + admin tooling
- Add a simple revocation CLI:
  - `wctl revoke-auth-token --jti <id>` (or `--subject <sub> --since <ts>`).
  - Stores revoked entries in Redis with TTL (exp - now).
- Add auth middleware checks for `jti` denylist in rq-engine and Flask helpers.
- Document rotation playbook (dual secrets or `WEPP_AUTH_JWT_SECRETS` list).
- Add a culvert JWT note to `docs/culvert-at-risk-integration/dev-package/README.md` (Bearer token, scopes, TTL guidance).

**Completed now:**
- `wctl revoke-auth-token` added (wrapper for `wepppy/weppcloud/_scripts/revoke_auth_token.py`).
- Revocation entries stored in Redis DB 0 (`auth:jwt:revoked:<jti>`) with TTL.
- rq-engine checks the denylist during JWT validation.
- Rotation playbook added to `docs/dev-notes/auth-token.spec.md`.

**Still pending:** none.

### Phase 7 - Webhook follow-on (tracked here, implemented later)
- Decide webhook registration surface (payload field vs `/rq-engine/api/webhooks`).
- Define payload schema + HMAC header (e.g., `X-WEPP-Signature`, `X-WEPP-Timestamp`, `X-WEPP-Event`).
- Store webhook config in `batch_metadata.json` or `culverts_runner.nodb`.
- Implement retry policy (exponential backoff, max attempts, final failure logged to batch root).
- Add tests with a mock HTTP endpoint and verify HMAC signatures.

## Culvert_app semi-permanent JWT guidance (MVP)
- Mint a service token scoped to `aud=rq-engine` with:
  - `scope=culvert:batch:submit culvert:batch:retry rq:status culvert:batch:read`
  - `sub=culvert-app` (or service identifier)
  - `service_groups=["culverts"]`
  - `jti=<uuid>` (required for revocation)
  - `expires-in` long TTL (target 90-180 days, confirm with security).
- Store in Culvert_web_app config and rotate when the TTL window elapses.

## Verification checklist
- rq-engine rejects missing/invalid tokens when `RQ_ENGINE_AUTH_MODE=required`.
- jobstatus/jobinfo remain read-only and open (no token required).
- culvert ingestion enforces `culvert:batch:submit`.
- Flask `/rq/api/*` is removed; session access is handled via rq-engine session tokens.
- revocation denylist blocks revoked `jti` values.
- `wctl issue-auth-token --json` prints token + claims and respects `--expires-in`.
- Tests added for rq-engine auth dependency + basic token validation paths.

## Open questions
- TTL for the culvert service token (90 vs 180 days) and rotation cadence.
- Should rq-engine use `aud=rq-engine` or a shared `aud=wepp-services`?
- Do we need multi-secret validation to support rotation (e.g., `WEPP_AUTH_JWT_SECRETS`)?
- Should anonymous jobstatus/jobinfo remain allowed in optional mode, or require tokens everywhere?
- Where should user-token authorization live for rq-engine (shared database access vs a thin authz service)?