Jon Chery
336 lines
14 KiB
Markdown
336 lines
14 KiB
Markdown
# Nova — v1.28 Research Findings
|
|
|
|
> Phase: research (pre-execution). Milestone: v1.28 (CLI Canonicalization
|
|
> + Identity Layer). Status: research. Researcher: ci-researcher.
|
|
> Autonomy: full.
|
|
>
|
|
> Research delegated to the ci-researcher subagent (full domain/ecosystem
|
|
> research with web citations). This file is the curated summary; the
|
|
> full 868-line research document is preserved in git history (the
|
|
> subagent's task output). Key findings + recommendations are below.
|
|
|
|
---
|
|
|
|
## §1 — Codebase Inventory (grounding)
|
|
|
|
### 1.1 `core/` modules (the REQ-324 subcommand surface)
|
|
|
|
19 Python files under `core/` (plus `core/lambda/`, `core/metrics/`).
|
|
Two already have `_cli.py` companions (`contract_resolver_cli.py` 40
|
|
lines, `regression_verify_cli.py` 32 lines) — the thin-delegate
|
|
precedent for `nova/<module>.py`. **No `nova/` dir, no `bin/`, no
|
|
`[project.scripts]` entry exists today.** The CLI is greenfield.
|
|
|
|
### 1.2 Existing Lambda pattern (`core/lambda/contract_ingestor.py`)
|
|
|
|
521 lines. Function URL + IAM auth (D-051). DynamoDB via lazy
|
|
module-global `boto3.resource`. Secrets Manager for tokens. Schema
|
|
validation in-Lambda. **`__main__` block already does CLI dispatch**
|
|
(`--check-readiness` → `core.submission_readiness.cli_main`) — this is
|
|
the dual-use precedent for REQ-329. Local testing via
|
|
`core/local_emulators.py:LocalLambdaStub`.
|
|
|
|
### 1.3 `core/env.py` — getter, not synthesizer
|
|
|
|
31 lines. `get_env(name, default)` reads `NOVA_<name>` from `os.environ`.
|
|
**REQ-330 needs a NEW `synthesize_local_env()` function** added here.
|
|
The closest existing pattern is `core/onboarding.py:generate_env_file()`.
|
|
|
|
### 1.4 `PolicyEngine` Protocol + `KyvernoJsonEngine` (the ABAC substrate)
|
|
|
|
`core/policy_engine.py`: `PolicyEngine` Protocol with `evaluate(payload,
|
|
policy_dir, contract_id) -> list[dict]`. `KyvernoJsonEngine` shells to
|
|
`kj scan --policy <dir> --payload <file> --output json`. Policy shape =
|
|
`ValidatingPolicy` (`apiVersion: json.kyverno.io/v1alpha1`) with
|
|
`spec.rules[].assert.all[].check` using JMESPath. Severity from
|
|
`metadata.annotations["nova.cloudinit.dev/severity"]`. **The payload
|
|
can be ANY JSON** — not just contracts (the v1.25 design point). This
|
|
is what makes kyverno-json usable for ABAC token vending (D-227).
|
|
|
|
### 1.5 `pyproject.toml` state
|
|
|
|
name `nova`, version `1.14.0`, requires-python `>=3.10` (spec wants
|
|
3.12 — bump needed for REQ-326). setuptools build backend. No
|
|
`[project.scripts]`, no `[tool.setuptools.packages.find]` — both needed.
|
|
Deps: `boto3`, `jsonschema`, `pyyaml`. No `argon2-cffi`, `cryptography`,
|
|
`pyjwt`, `click`/`typer` — **argparse-only** is the repo convention.
|
|
|
|
### 1.6 Forge conventions
|
|
|
|
`.github/workflows/` + `.gitea/workflows/` kept byte-identical. Python
|
|
3.12 already pinned via `actions/setup-python@v5`. No composite action
|
|
exists yet — `nova cli-action` (REQ-326) is greenfield.
|
|
|
|
### 1.7 IAM baseline (load-bearing for REQ-340)
|
|
|
|
`.ciagent/IAM_POLICY.md` + `terraform/bootstrap/spike_runner_policy.json`.
|
|
The `nova-spike-runner` principal already has KMS (incl. `CreateKey`,
|
|
`Sign`, `GetPublicKey`), Lambda (incl. `PublishLayerVersion`), DynamoDB
|
|
grants. **New grants needed:** `cloudformation:*` (for `nova idp setup
|
|
--apply`) + `codeartifact:*` (for the wheel publish pipeline). Flagged
|
|
for P1/P2.
|
|
|
|
---
|
|
|
|
## §2 — CodeArtifact + Lambda Layer Pipeline (REQ-323)
|
|
|
|
**Recommendation:** single CI job on merge to `main` affecting
|
|
`core/**`/`adapters/**`/`nova/**`/`pyproject.toml`. Build wheel
|
|
(`python -m build --wheel`) → `twine upload` to CodeArtifact → build
|
|
layer (`pip install --target layer/python/ dist/nova-*.whl argon2-cffi
|
|
cryptography pyjwt`) → `aws lambda publish-layer-version` → record
|
|
version mapping in SSM `/nova/layer/nova-cli/version` (CAP-035). If
|
|
either publish fails, the job fails (merge blocked, REQ-323 AC).
|
|
|
|
**Atomicity:** wheel publish is idempotent (pin version to
|
|
`<semver>+<sha7>`); layer publish retries on failure. CAP-035 reads the
|
|
SSM parameter to verify layer-version ↔ wheel-version match.
|
|
|
|
**Risks:** CodeArtifact not yet provisioned in `581513795199` (CLARIFY
|
|
assumption #1); `codeartifact:*` grant missing. Fallback: Gitea-hosted
|
|
wheel index. Layer `--compatible-architectures`: build x86_64 only for
|
|
v1.28 (aarch64 only if Graviton Lambda needed).
|
|
|
|
---
|
|
|
|
## §3 — CLI Subcommand Architecture (REQ-324)
|
|
|
|
**Recommendation:** three-layer. `nova/__init__.py` (marker) →
|
|
`nova/cli.py` (~80 lines, auto-discovers `nova/<module>.py` via
|
|
`pkgutil.iter_modules`, dispatches, emits `cli.invocation` audit event)
|
|
→ `nova/<module>.py` (≤50 lines each, exports `add_parser(subparsers)`
|
|
+ `run(args) -> int`, delegates to `core/`). Entry point:
|
|
`[project.scripts] nova = "nova.cli:main"`. **argparse-only** (no
|
|
click/typer — repo convention).
|
|
|
|
**CAP-034 AST scan:** ≤50 lines; ≤3 function defs; every `ast.Call`
|
|
resolves to a `core.` import; no conditionals beyond `if __name__`.
|
|
|
|
**Subcommand groups:** `nova auth`, `nova idp`, `nova metrics` =
|
|
nested subparsers (same pattern, one level deeper).
|
|
|
|
**setuptools:** add `[tool.setuptools.packages.find]` including `nova`,
|
|
`nova.*`, `core`, `core.*`, `adapters.*`.
|
|
|
|
---
|
|
|
|
## §4 — Argon2id in Lambda Python 3.12 (REQ-334, D-228)
|
|
|
|
**Findings:** `argon2-cffi-bindings` v25.1.0 ships `cp39-abi3`
|
|
manylinux x86_64 + aarch64 wheels — **ABI-stable, compatible with
|
|
Python 3.9..3.13**. Lambda Python 3.12 runs Amazon Linux 2023 (glibc
|
|
2.34 ≥ 2.28 required). **The abi3 manylinux wheel loads cleanly.**
|
|
Confidence: 0.92.
|
|
|
|
**D-228 AMENDMENT:** the "pure-Python fallback" clause is **weaker than
|
|
stated** — there is no maintained pure-Python Argon2 implementation. A
|
|
pure-Python crypto fallback is a **liability** (weaker hashing,
|
|
violates INV-16's spirit). Revised recommendation:
|
|
1. **Primary:** bundled manylinux abi3 wheel in the `nova-cli` Lambda
|
|
layer. Works. Confidence 0.92.
|
|
2. **Fallback:** detect `ImportError` at Lambda cold-start → **fail
|
|
closed** (503, refuse sign-ups). The Lambda health check reports
|
|
C-extension status. **Do NOT ship a pure-Python fallback.**
|
|
3. **Escape hatch:** Fargate (~1 week, per CLARIFY Q1).
|
|
|
|
Lambda memory ≥ 512 MB (Argon2id memory_cost ~20 MB + overhead).
|
|
|
|
---
|
|
|
|
## §5 — KMS Asymmetric Signing for OIDC Tokens (REQ-337)
|
|
|
|
**Recommendation: key spec = `ECC_NIST_P256`, alg = `ECDSA_SHA_256`
|
|
(JWS `ES256`).** RSA-2048 is larger + slower; P-256 is RFC 7518's
|
|
recommended JWT alg. Signature size 64 bytes (vs RSA 256). JWKS
|
|
compactness matters (fetched often).
|
|
|
|
**The #1 gotcha:** KMS returns DER-encoded ECDSA signatures; **JWS
|
|
requires raw r‖s concatenation** (RFC 7515 §3.1.3). The token-vend
|
|
Lambda converts via `cryptography.hazmat.primitives.asymmetric.utils.
|
|
decode_dss_signature` → `r.to_bytes(32) + s.to_bytes(32)`. ~5 lines.
|
|
Flagged for the threat model (REQ-347) + KMS round-trip test (REQ-350).
|
|
|
|
**Flow:** validate PAT → ABAC eval → build JWT header/payload →
|
|
`kms.sign(Message=signing_input, MessageType="RAW", SigningAlgorithm=
|
|
"ECDSA_SHA_256")` → DER→raw → JWT. `kid` = KMS key alias.
|
|
|
|
**Verification:** use `pyjwt` (`jwt.decode` handles JWK→key natively);
|
|
`cryptography` only for SPKI→JWK in the JWKS Lambda.
|
|
|
|
**Rotation:** manual, 90 days (matches D-069 CMK cadence). New key +
|
|
re-point alias + JWKS serves both `kid`s during overlap.
|
|
|
|
---
|
|
|
|
## §6 — JWKS Endpoint (REQ-338, D-230)
|
|
|
|
**D-230 confirmed.** Lambda function URL (`AuthType: NONE` — JWKS is
|
|
public-key only) + reserved concurrency 10 (max 100 RPS, JWKS is
|
|
cached client-side). `Cache-Control: max-age=3600`. Separate tiny
|
|
`nova-idp-jwks` Lambda (separation of concerns).
|
|
|
|
**Custom domain + WAF = OPTIONAL** via `--public-jwks-domain <domain>`
|
|
flag on `nova idp setup`. Without it, raw function URL (acceptable for
|
|
v1.28 pilot). With it: CloudFront + ACM + WAF rate-based rule (>100
|
|
req/5min per IP) + Route53 ALIAS. Adds ~8 CloudFormation resources.
|
|
|
|
**Defer API Gateway** (D-230) — $3.50/M + complexity for no benefit at
|
|
v1.28 volume.
|
|
|
|
---
|
|
|
|
## §7 — kyverno-json ABAC Policy (REQ-339, D-227)
|
|
|
|
**D-227 confirmed.** Policy at `platform/abac/token-vend.policy` =
|
|
`ValidatingPolicy` with JMESPath checks against a payload of
|
|
`{subject, requested_claims, target_resource, environment, pat_jti,
|
|
policy_version}`. Decision logic: any `fail` PCR with severity
|
|
`critical` → deny (403 + audit); all pass → allow → KMS sign.
|
|
|
|
**`policy_version` (D-231):** git SHA of the policy file, baked into
|
|
the Lambda layer, recorded in every `token.vend.allowed/denied` audit
|
|
event.
|
|
|
|
**BIGGEST PACKAGING RISK:** the token-vend Lambda needs the `kj` Go
|
|
binary (~40 MB) on PATH. Bundle it in the `nova-cli` Lambda layer
|
|
(`wget` the Linux amd64 release into `layer/bin/kj`). `KyvernoJsonEngine
|
|
.is_configured()` checks `which kj` → `/opt/bin/kj` (layer mount). P2
|
|
spike confirms it runs in AL2023 Lambda. Fallback: Fargate. Confidence
|
|
0.75 — needs the spike.
|
|
|
|
---
|
|
|
|
## §8 — PAT Lifecycle (REQ-342, REQ-343, REQ-344)
|
|
|
|
**PAT = signed JWT** (KMS-signed, `typ: "developer_pat"` distinguishes
|
|
from `nova_oidc_token` per INV-14). Claims: `iss, sub, typ, jti, iat,
|
|
exp, roles, owner`.
|
|
|
|
**`nova-pats` DynamoDB table** (4th table): PK=`jti`, GSI1=`sub` (list
|
|
PATs for user), GSI2=`pat_hash` (lookup by hash). Only the hash stored
|
|
(not raw PAT). Revoked PATs retained for audit.
|
|
|
|
**Revocation (D-229 CLARIFIED):** GSIs don't support strongly-consistent
|
|
reads. The token-vend Lambda extracts `jti` from the PAT JWT (decode
|
|
without verifying — signature verified separately) →
|
|
`GetItem(PK=jti, ConsistentRead=True)` on the main table. Satisfies the
|
|
60s SLO. Confidence 0.90.
|
|
|
|
**CLI:** `nova auth login` (session→OIDC token, store locally),
|
|
`nova auth revoke --pat <jti>`, `nova auth status` (active credential,
|
|
mode, selection_reason). Local file `~/.nova/credentials.json` (0600,
|
|
never to stdout, in `.gitignore`). "Most recent wins" (D-226 Q5) =
|
|
`active_credential_jti` field.
|
|
|
|
---
|
|
|
|
## §9 — `nova idp setup` CloudFormation (REQ-340, REQ-341)
|
|
|
|
**Template (raw dict → JSON, no troposphere dep):** 2-3 Lambdas, 4
|
|
DynamoDB tables (`nova-users`, `nova-sessions`, `nova-password-resets`,
|
|
`nova-pats`), KMS key `alias/nova-oidc-signing` (ECC_NIST_P256),
|
|
function URLs, IAM roles, optional CloudFront/WAF/ACM.
|
|
|
|
**`--check`:** validates prerequisites (AWS creds, CFN perms, KMS perms,
|
|
layer exists via CAP-035). Prints required IAM policy delta.
|
|
**`--apply`:** generate → print to temp file + resource summary →
|
|
`$PAGER` → `Apply? [y/N]` → `cloudformation deploy --capabilities
|
|
CAPABILITY_IAM`. NFR-10 satisfied by the explicit prompt.
|
|
**`--dry-run`:** resource list only, no write.
|
|
**`--verify`:** runs the KMS round-trip test (REQ-350).
|
|
|
|
**New IAM grants needed:** `cloudformation:*`, `iam:CreateRole`/`PassRole`,
|
|
`lambda:CreateFunction`/`CreateFunctionUrlConfig`,
|
|
`dynamodb:CreateTable`, `kms:CreateKey`/`CreateAlias`, `ssm:PutParameter`.
|
|
|
|
---
|
|
|
|
## §10 — GitHub + Gitea Marketplace Composite Action (REQ-326)
|
|
|
|
**Single `action.yml`** at `.github/actions/nova-cli/action.yml`,
|
|
referenced by both GitHub + Gitea via `uses: continuous-intelligence/
|
|
acdl/.github/actions/nova-cli@v1.28`. Composite action: `setup-python@v5`
|
|
(python 3.12) → CodeArtifact login + `pip install nova` → `nova
|
|
${{ inputs.command }}`. `NOVA_CLIENT_MODE` env from input.
|
|
|
|
**Byte-identical test (REQ-326 AC2):** CI matrix runs the action on
|
|
GitHub `ubuntu-latest` + Gitea `act_runner` with same inputs; assert
|
|
same stdout/exit code.
|
|
|
|
**Risk:** Gitea `actions/checkout`/`setup-python` may need Gitea
|
|
mirrors (`https://gitea.com/actions/...`). P1 test on the actual Gitea
|
|
instance. Confidence 0.70.
|
|
|
|
---
|
|
|
|
## §11 — `mode_resolver` Priority (REQ-327, D-226)
|
|
|
|
**TTY detection: check `sys.stdin.isatty()`** (NOT stdout). Edge 3
|
|
(`nova apply | tee log.txt`): stdout piped, stdin is TTY → user is
|
|
present → `interactive` (correct). `sys.stdout.isatty()` would
|
|
misresolve to `agent`. **`stdin` answers "is a human at a terminal?"**
|
|
|
|
**Credential type detection:** read `~/.nova/credentials.json` →
|
|
`active_credential_jti`'s `type` (`developer_pat`/`nova_oidc_token`).
|
|
Both + TTY → `interactive`; + no TTY → `agent` (INV-14).
|
|
|
|
**Property tests (REQ-349):** `hypothesis` with strategies for
|
|
flag/env/cred/tty. Properties: deterministic (INV-13), flag-wins,
|
|
invalid-env-ignored, no-silent-fallback (every resolution has a
|
|
non-empty `selection_reason`).
|
|
|
|
**`mode_resolver.py` lives in `core/`** (not `nova/`) so Lambdas could
|
|
import it, but **it's CLI-only** — the token-vend Lambda doesn't resolve
|
|
modes.
|
|
|
|
---
|
|
|
|
## §12 — Persona Assessment
|
|
|
|
See `.ciagent/PERSONAS.md` for the full YAML roster. Summary:
|
|
- **Deactivate** frontend-engineer (no UI) + data-engineer (no data
|
|
pipelines in v1.28).
|
|
- **Activate** backend-engineer (Lambda/DynamoDB/KMS/CodeArtifact) +
|
|
lead-developer (plan/review/ship).
|
|
- **Add** security-engineer (Argon2id/KMS/ABAC/threat model) +
|
|
cli-engineer (subcommand surface/mode_resolver/argparse/CAP-034).
|
|
|
|
---
|
|
|
|
## §13 — Architecture Sketch (ARCHITECTURE.md §12.10)
|
|
|
|
See `.ciagent/ARCHITECTURE.md` §12.10 (appended this stage). New
|
|
greenfield files: `nova/` CLI package, `platform/abac/token-vend.policy`,
|
|
`core/mode_resolver.py`, `core/env.py:+synthesize_local_env()`,
|
|
`core/lambda/nova_idp_{auth,token_vend,jwks}.py`, `tests/test_*`,
|
|
`docs/{operator-guide-idp,developer-guide-auth,threat-model}.md`.
|
|
|
|
---
|
|
|
|
## Decisions re-validated / amended
|
|
|
|
| Decision | Status | Change |
|
|
|---|---|---|
|
|
| D-226 | re-validated + refined | `sys.stdin.isatty()` is the TTY check (not stdout) |
|
|
| D-227 | re-validated | `kj` Go binary bundled in Lambda layer — packaging risk flagged |
|
|
| D-228 | **amended** | Pure-Python fallback → fail-closed + Fargate (pure-Python crypto is a liability) |
|
|
| D-229 | re-validated + clarified | Strong read on main table PK (`jti`), not GSI (GSIs don't support strong reads) |
|
|
| D-230 | re-validated | CloudFront/WAF/ACM made optional via `--public-jwks-domain` flag |
|
|
| D-231 | re-validated | `policy_version` (git SHA) in the ABAC payload |
|
|
|
|
**New recommendations for PLAN/GRILL to formalize (no D-ID yet):**
|
|
- KMS key spec = `ECC_NIST_P256`, alg `ES256`; DER→raw ECDSA conversion required.
|
|
- `nova-cli` Lambda layer bundles the `kj` Go binary (~40 MB).
|
|
- `nova-pats` = 4th DynamoDB table; PK=`jti`, GSI1=`sub`, GSI2=`pat_hash`.
|
|
- `sys.stdin.isatty()` is the TTY heuristic.
|
|
- `[project.scripts] nova = "nova.cli:main"`; argparse-only.
|
|
- `cloudformation:*` + `codeartifact:*` = new IAM baseline grants (P1/P2).
|
|
|
|
---
|
|
|
|
## RESEARCH complete
|
|
|
|
All 11 research questions answered with cited findings + concrete
|
|
recommendations + risks. D-228 amended (fail-closed, not pure-Python
|
|
fallback). The `kj` binary packaging is the highest-risk item (P2
|
|
spike). Next: PLAN. |