acdl/docs/operator-guide-idp.md

Operator Guide — Nova IdP Setup (nova idp setup)

REQ-345 — operator guide for nova idp setup. Covers --check, --apply, --verify, the prerequisite IAM policy, the CloudFormation review flow, and the C-6.3 grill additions: KMS key rotation (90 days), Lambda layer update, DDB PITR restore, emergency PAT revocation (DDB-level, not CLI).

Audience: platform operators / SREs deploying the Nova identity stack into AWS account 581513795199 (or a fresh account). No developer auth flows here — see docs/developer-guide-auth.md for those.

1. Overview

nova idp setup provisions the Nova identity layer (Nova-idp) as a CloudFormation stack. The stack contains:

Resource	Count	Notes
Lambda functions	3	nova-idp-auth, nova-idp-token-vend, nova-idp-jwks
DynamoDB tables	4	nova-users, nova-sessions, nova-password-resets, nova-pats (PITR enabled on each, REQ-335)
KMS asymmetric key	1	alias/nova-oidc-signing (ECC_NIST_P256, SIGN_VERIFY)
Lambda function URLs	3	auth + token-vend (IAM auth), jwks (AuthType: NONE)
IAM roles	3+	one per Lambda + the CloudFormation service role
Optional CloudFront + WAF + ACM	0/3	only with --public-jwks-domain

The command has three modes — --check, --apply, --verify — plus --dry-run for a resource-only preview. All modes are safe to re-run.

2. nova idp setup --check

Run before --apply to verify the deploying principal has the permissions and environment the stack needs.

nova idp setup --check

What it checks

1. AWS credentials — aws sts get-caller-identity succeeds and returns an Account id. If this fails, run aws configure or export AWS_PROFILE / AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY.
2. AWS region — AWS_DEFAULT_REGION or AWS_REGION is set. The stack is regional (single-region); pick the region you want all resources to live in.
3. CloudFormation permissions — the principal can create/describe stacks (see §5 for the full IAM delta).
4. KMS permissions — kms:CreateKey + kms:CreateAlias (needed to mint alias/nova-oidc-signing).
5. Lambda layer exists — the nova-cli Lambda layer (published by the P1 Wave 4 pipeline) is referenced by the stack; --check reports whether the layer ARN in SSM (/nova/layer/nova-cli/version) is present. If absent, run the publish workflow or nova layer update.

Reading the IAM policy delta

--check prints a report like:

{
  "aws_creds": true,
  "region": "us-east-1",
  "missing": [],
  "iam_delta": [
    "cloudformation:*",
    "iam:CreateRole",
    "iam:PassRole",
    "lambda:CreateFunction",
    "lambda:CreateFunctionUrlConfig",
    "dynamodb:CreateTable",
    "kms:CreateKey",
    "kms:CreateAlias"
  ]
}

iam_delta is the delta between what the deploying principal currently has (the nova-spike-runner grants in this account) and what --apply needs. Each entry is a grant you must add to the principal's policy before --apply will succeed. --check never makes changes.

3. nova idp setup --apply

Generates the CloudFormation template, presents it for review, and deploys only after explicit y/N approval (NFR-10).

nova idp setup --apply

Review flow

1. Resource summary printed to stdout (resource type → count):

   Resource summary:
     AWS::DynamoDB::Table: 4
     AWS::IAM::Role: 3
     AWS::KMS::Key: 1
     AWS::Lambda::Function: 3
     AWS::Lambda::Url: 3
   

2. Full template opened in $PAGER (if set and stdin is a TTY); otherwise the path to the temp file is printed. Review every resource, especially the KMS key policy and the IAM roles.
3. Apply? [y/N] prompt. Type y + Enter to deploy; anything else aborts. No resource is created before this approval.
4. On approval: aws cloudformation deploy --stack-name nova-idp --template-file <tmp> --capabilities CAPABILITY_IAM.

--dry-run — resource list only

nova idp setup --dry-run

Generates the template and prints the resource summary without the pager, the prompt, or any deploy. Use this to audit the stack shape in CI or before a manual --apply.

--public-jwks-domain — optional custom domain + WAF

nova idp setup --apply --public-jwks-domain jwks.nova.example.com

Adds a CloudFront distribution fronting the JWKS Lambda function URL, an ACM certificate (DNS-validated) for the domain, and a WAF web ACL with a rate-based rule (see §C-6.3 and the threat model). Without this flag the JWKS endpoint is a bare function URL (AuthType: NONE) — fine for piloting but exposed to the internet without rate limiting. For any public deployment, set --public-jwks-domain.

4. nova idp setup --verify

Runs the KMS round-trip test (CAP-037) against the deployed stack.

nova idp setup --verify

It signs a test JWT via core.kms_signing.sign_jwt() (using the real KMS key alias/nova-oidc-signing), fetches the JWKS endpoint, and verifies the JWT signature with pyjwt + the JWKS key. This exercises the full DER → raw ECDSA conversion path (the #1 implementation risk — see docs/threat-model.md).

Success output:

{"passed": true, "detail": "KMS round-trip OK"}

Failure output:

{"passed": false, "detail": "verify error: <exception>"}

Common failure causes:

• The KMS key policy doesn't grant kms:Sign to the verify caller.
• The JWKS function URL is not deployed or returns a non-200.
• The KMS key spec isn't ECC_NIST_P256 (the DER→raw conversion assumes P-256, 32-byte coordinates).

5. Required IAM policy

The delta --check reports is the set of grants the deploying principal needs in addition to the existing nova-spike-runner grants. The full required set:

Action	Why
cloudformation:*	create/deploy/describe the nova-idp stack
codeartifact:*	(already on nova-spike-runner) publish the wheel + layer
iam:CreateRole	create the per-Lambda execution roles
iam:PassRole	pass those roles to Lambda + CloudFormation
lambda:CreateFunction	create the 3 Lambda functions
lambda:CreateFunctionUrlConfig	create the 3 function URLs
dynamodb:CreateTable	create the 4 DDB tables (with PITR)
kms:CreateKey	mint the ECC_NIST_P256 signing key
kms:CreateAlias	bind alias/nova-oidc-signing to the key
ssm:PutParameter	write the layer-version mapping to SSM

Attach these to the deploying principal's policy before --apply. --check will then report an empty missing list.

---

C-6.3 Grill additions — operational runbooks

The grill (C-6.3) requires four operational procedures beyond the setup flow. Each is a runbook an on-call SRE can follow without reading source code.

6. KMS key rotation (90-day cadence)

Cadence: rotate alias/nova-oidc-signing every 90 days. The rotation is a key re-point, not a key deletion — the alias is moved to a new key while the old key stays valid during the token-overlap window so already-issued tokens keep verifying.

Procedure:

1. Create the new key (same spec):

   NEW_KEY=$(aws kms create-key \
     --key-spec ECC_NIST_P256 \
     --key-usage SIGN_VERIFY \
     --description "nova-oidc-signing-$(date +%Y%m%d)" \
     --query KeyId --output text)
   

2. Re-point the alias to the new key:

   aws kms update-alias --alias-name alias/nova-oidc-signing \
     --target-key-id "$NEW_KEY"
   

3. JWKS serves both kids during the overlap window. The JWKS Lambda lists all keys the alias has pointed at that are still enabled. Already-issued OIDC tokens (signed with the old key) keep verifying until they expire (OIDC TTL default 15 min; PAT TTL ≤ 24h dev / ≤ 1h service-account). Do not disable the old key until at least the max PAT TTL (24h) has elapsed.
4. After the overlap window (≥ 24h), disable + schedule deletion of the old key:

   aws kms disable-key --key-id "<old-key-id>"
   aws kms schedule-key-deletion --key-id "<old-key-id>" --pending-window-in-days 7
   

5. Verify the new key is active:

   nova idp setup --verify
   

Audit: emit a manual kms.key_rotated event to the audit stream with old_key_id, new_key_id, rotated_at. The rotation is a CloudFormation-less operation (KMS aliases are mutable); it does not require a stack update.

7. Lambda layer update

The nova-cli Lambda layer (the shared dependency bundle: argon2-cffi, cryptography, pyjwt, kj binary) is republished automatically on every merge to main by the P1 Wave 4 publish workflow (the byte-identical GitHub + internal-forge workflow files). On a successful publish, the new layer version ARN is written to SSM /nova/layer/nova-cli/version.

When to update manually:

• A dependency CVE requires an out-of-band patch before the next merge.
• The kj binary pinned version changes (C-8.2 supply-chain safety).

Manual procedure:

nova layer update

This rebuilds the layer (pip install --target layer/python/ + the pinned kj binary, SHA256 verified against layer/kj.sha256), publishes a new lambda:PublishLayerVersion, and updates the SSM parameter. The 3 Nova-idp Lambdas pick up the new layer on their next cold start (or force a redeploy with aws lambda update-function-configuration --layers <new-arn> on each).

Verify: nova idp setup --verify after the Lambdas reload.

8. DynamoDB PITR restore

All 4 identity tables have point-in-time recovery (PITR) enabled (REQ-335): nova-users, nova-sessions, nova-password-resets, nova-pats. PITR lets you restore a table to any second in the last 35 days (the AWS retention window).

Procedure (restore nova-pats to 1 hour ago):

# 1. Find the restore target time (ISO 8601, UTC, within the last 35d).
RESTORE_TO=$(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%SZ)

# 2. Restore to a NEW table (PITR never overwrites the source).
aws dynamodb restore-table-to-point-in-time \
  --source-table-name nova-pats \
  --target-table-name nova-pats-restored \
  --restore-date-time "$RESTORE_TO" \
  --billing-mode-restore-as-is

# 3. After the restore completes (status ACTIVE), repoint the app:
#    - update the stack env var NOVA_PATS_TABLE=nova-pats-restored, or
#    - rename: delete nova-pats, then aws dynamodb update-table --table-name
#      nova-pats-restored --new-table-name nova-pats (downtime window).
# 4. Re-enable PITR on the restored table (PITR does not carry over).
aws dynamodb update-continuous-backups \
  --table-name nova-pats-restored \
  --point-in-time-recovery-specification PointInTimeRecoveryEnabled=true

Which tables have PITR: all 4 (nova-users, nova-sessions, nova-password-resets, nova-pats). Verify with:

for t in nova-users nova-sessions nova-password-resets nova-pats; do
  aws dynamodb describe-continuous-backups --table-name "$t" \
    --query 'ContinuousBackupsDescription.PointInTimeRecoveryDescription' --output text
done

Recovery window: 35 days (AWS PITR). Restores older than 35 days are impossible — for longer retention, export to S3 via the on-demand export or a scheduled AWS Backup plan.

9. Emergency PAT revocation (DDB-level, not CLI)

When to use: a PAT is known-compromised and the nova auth revoke CLI is unavailable (e.g. the operator machine is offline, or the PAT jti is known but the raw PAT is not — revocation is keyed on jti, not the token string). This is a DDB-level operation; it bypasses the CLI but still satisfies the D-229 strong-read SLO (the token-vend Lambda does a ConsistentRead=True GetItem on jti on every vend — the revocation is reflected on the next vend, within 60s P95).

Procedure:

aws dynamodb update-item \
  --table-name nova-pats \
  --key '{"jti":{"S":"<jti>"}}' \
  --update-expression "SET #s = :r" \
  --expression-attribute-names '{"#s":"status"}' \
  --expression-attribute-values '{":r":{"S":"revoked"}}'

Replace <jti> with the PAT's jti claim (a uuid4; find it in the pat.issued audit event or by scanning the sub-index GSI for the compromised subject). The item is retained (not deleted) so the audit trail is intact — only status flips from active to revoked.

Verify the revocation took effect:

aws dynamodb get-item \
  --table-name nova-pats \
  --key '{"jti":{"S":"<jti>"}}' \
  --consistent-read \
  --query 'Item.status.S' --output text
# → revoked

The next token-vend call with that jti returns 403 pat_revoked immediately (D-229: the strong read is synchronous).

Bulk revocation (revoke all of a subject's PATs):

SUB="<sub>"
JTIS=$(aws dynamodb query \
  --table-name nova-pats \
  --index-name sub-index \
  --key-condition-expression "sub = :s" \
  --expression-attribute-values "{\":s\":{\"S\":\"$SUB\"}}" \
  --query 'Items[?status.S==`active`].jti.S' --output text)
for jti in $JTIS; do
  aws dynamodb update-item --table-name nova-pats \
    --key "{\"jti\":{\"S\":\"$jti\"}}" \
    --update-expression "SET #s = :r" \
    --expression-attribute-names '{"#s":"status"}' \
    --expression-attribute-values '{":r":{"S":"revoked"}}'
done

---

Appendix — quick reference

Command	What it does
nova idp setup --check	prerequisites + IAM delta (no changes)
nova idp setup --dry-run	resource summary only (no deploy)
nova idp setup --apply	review template → y/N → deploy
nova idp setup --apply --public-jwks-domain <fqdn>	add CloudFront + WAF + ACM
nova idp setup --verify	KMS round-trip test (CAP-037)

Runbook	Cadence / trigger
KMS key rotation	every 90 days
Lambda layer update	on merge (auto) or manually via nova layer update
DDB PITR restore	on data loss / corruption (35-day window)
Emergency PAT revocation	on compromise (DDB-level, immediate)