Jon Chery
541 lines
35 KiB
Markdown
541 lines
35 KiB
Markdown
# Nova — The New Dawn of DevSecOps
|
||
|
||
> **Compressed.** The full v1.0–v1.24 milestone-by-milestone narrative is
|
||
> preserved verbatim at `.ciagent/archive/PROJECT-v1.0-v1.24.md`. This file
|
||
> retains only the durable vision/tenets/scope, the still-load-bearing
|
||
> decisions (D-034..D-072, W1.A..BA.F, Q1.3), the capability status, and
|
||
> the active milestone (v1.26) + its immediate predecessor (v1.25).
|
||
>
|
||
> **Rebrand complete (milestone v1.15 — Nova, tag v1.15.4).** The project
|
||
> was rebranded from **ACDL** / "Agentic Cloud Delivery Platform" →
|
||
> **Nova** / "The New Dawn of DevSecOps — security as a seamless enabler
|
||
> of fast deployments."
|
||
|
||
## Vision / Core Value
|
||
|
||
Consumers declare intent; the platform delivers safe production
|
||
deployment through an agentic stack. The platform absorbs two frictions:
|
||
the cognitive load of getting the infrastructure right, and the
|
||
operational work of getting the change to production safely.
|
||
|
||
Source of truth for **why**: `docs/vision.md`.
|
||
Source of truth for **how**: `docs/architecture.md` + `.ciagent/ARCHITECTURE.md`.
|
||
Where the two conflict, the vision wins.
|
||
|
||
## North Star
|
||
|
||
A merged change progresses through lower environments end-to-end without a
|
||
platform engineer joining a thread, approving a ticket, or manually
|
||
triggering a stage gate. A non-technical consumer ships a production
|
||
deployment by declaring intent — without authoring a workflow, a
|
||
configuration file, or a Terraform module. Every production change is
|
||
traceable to a human attestation and an immutable evidence stream.
|
||
|
||
## Core Tenets (from `docs/vision.md`)
|
||
|
||
1. **Operations are Declared, Not Executed.** Consumers define what they
|
||
need; the platform reconciles, provisions, and progresses.
|
||
2. **The Delivery Lifecycle is a Sovereign Boundary.** The platform
|
||
governs infra and delivery; it does not reach into upstream product/SDLC.
|
||
Integration is only through validated, published contracts.
|
||
3. **Lower Environments are Autonomous; Higher Environments are Attested.**
|
||
Dev = zero-touch agentic. QA/prod/dr = deliberate human attestation, not
|
||
rubber stamps.
|
||
4. **Safety is Computed, Not Assumed.** Every action produces a measurable,
|
||
explainable confidence signal.
|
||
5. **Infrastructure is Consumed, Not Maintained.** No node/OS/bare-metal lifecycle.
|
||
6. **Two Consumer Surfaces, One Platform.** Technical developers (L3A) and
|
||
non-technical consumers (L3B) converge on the same contract schema, the
|
||
same policy envelope, and the same evidence stream.
|
||
|
||
## Domain Boundaries
|
||
|
||
- **In scope:** environment progression; cloud resource lifecycle; operational
|
||
security and observability NFRs; policy enforcement; immutable audit
|
||
lineage; confidence frameworks; two consumer surfaces (developer + agentic).
|
||
- **Out of scope:** application business logic; IDE workflows; product
|
||
backlog / sprint planning; compute requiring node-level or OS-level management.
|
||
- **Interface:** upstream systems integrate through a strict contract
|
||
boundary. The platform validates, enriches with operational standards,
|
||
and reconciles the target state.
|
||
|
||
## Scope: Nova is Downstream of PDLC
|
||
|
||
> Promoted from Core Tenet #2 + Anti-Goal #1 (v1.18, REQ-216).
|
||
|
||
The **Product Development Lifecycle (PDLC)** — product backlog, code
|
||
authorship, IDE workflows, sprint planning, application business logic —
|
||
is **upstream** of Nova. Nova never reaches into the PDLC. Nova's domain is
|
||
**infrastructure + delivery only**: environment progression, cloud
|
||
resource lifecycle, operational security/observability NFRs, policy
|
||
enforcement, immutable audit lineage, and the two consumer surfaces.
|
||
|
||
Integration between the PDLC and Nova is **only** through the validated,
|
||
published contract boundary (`schemas/contract.schema.json` +
|
||
`schemas/submission-readiness.schema.json`). The citizen developer's AI
|
||
coding agent, an upstream agentic SDLC platform, or any upstream
|
||
development platform may all produce submissions — the source does not
|
||
matter because all are subject to the same compliance standards (the
|
||
submission-readiness gate, D-133).
|
||
|
||
```
|
||
PDLC (upstream) Nova (downstream)
|
||
───────────────── ─────────────────
|
||
product backlog contract ingestion
|
||
code authorship (AI agent / IDE / SDLC) → submission-readiness gate
|
||
sprint planning → policy enforcement
|
||
application business logic → cloud resource lifecycle
|
||
→ environment progression (dev→qa→prod→dr)
|
||
→ immutable audit + attestation
|
||
```
|
||
|
||
## RACI Matrix
|
||
|
||
> Source of truth (v1.18, REQ-215, D-139).
|
||
|
||
### Roles
|
||
|
||
- **Citizen Developer (CD)** — the consumer (technical developer L3A or
|
||
non-technical L3B). Responsible for all **Functional Requirements (FRs)**
|
||
and **User Acceptance Testing (UAT)**. The FRs + UAT may originate from
|
||
any upstream source — all subject to the same compliance standards (the
|
||
submission-readiness gate, D-133).
|
||
- **Platform** — Nova. Responsible for all **Non-Functional Requirements
|
||
(NFRs)**, **Infrastructure** (cloud resource lifecycle, state, IAM),
|
||
**QA** (platform-side quality checks: policy, confidence, schema), and
|
||
**Production deployments to cloud**.
|
||
- **Release Management (RM)** — **co-owned**. QA + SRE attestations are
|
||
required by the actual release. The platform performs the checks
|
||
agentically; the citizen developer authorizes (the human attestation at
|
||
the stage gate, D-042, `hitl_gates.py`).
|
||
|
||
### Matrix
|
||
|
||
| Work Category | Citizen Developer | Platform | Release Management |
|
||
|---|---|---|---|
|
||
| **Functional Requirements (FRs)** | **R/A** | C | I |
|
||
| **User Acceptance Testing (UAT)** | **R/A** | C | I |
|
||
| **Non-Functional Requirements (NFRs)** | I | **R/A** | C |
|
||
| **Infrastructure (cloud, state, IAM)** | I | **R/A** | C |
|
||
| **QA (policy, confidence, schema checks)** | C | **R/A** | I |
|
||
| **Production deployment to cloud** | I | **R/A** | C |
|
||
| **Release attestation (QA + SRE sign-off)** | **A** | R | **R** |
|
||
|
||
**Key: R** = Responsible · **A** = Accountable · **C** = Consulted · **I** = Informed.
|
||
|
||
The release is co-owned: the platform runs the checks; the citizen
|
||
developer authorizes the promotion. This is the "autonomy in operations,
|
||
human at stage gates" model from the NORTH_STAR.
|
||
|
||
## Capability Status (Re-Verified 2026-07-27)
|
||
|
||
> **PO-facing capability catalog:** `.ciagent/STATE.md` (additive;
|
||
> updated at milestone ship). CAP-NNN IDs cross-reference the regression
|
||
> gate at `core/regression_verify.py`.
|
||
> Source of truth (the 2026-07-27 sweep, archived v1.27):
|
||
> `.ciagent/archive/CAPABILITY_INVENTORY-v1.10.md` (Phase 54, D-093).
|
||
> Tier: **local** = runs via emulating adapters (no AWS); **live-aws** =
|
||
> runs against the live AWS account (581513795199).
|
||
|
||
**Decay disclosure.** Capabilities marked complete in v1.1–v1.8 were not
|
||
reproducible as of 2026-07-27 (7 adapter defects). The v1.10 milestone
|
||
(Phases 52–55) re-verified every advertised capability and fixed all 7
|
||
defects in-sweep (D-090: no cap). The headline E2E now passes at both tiers.
|
||
|
||
**Auto-verified capabilities (16/16 Verified):**
|
||
|
||
| ID | Capability | Tier | Status |
|
||
|----|-----------|------|--------|
|
||
| CAP-001..CAP-012 | contract schema, resolver, adapter, interpolation, confidence, outbox, pytest, run_ci, local E2E (microservice + static-assets) | local | Verified |
|
||
| CAP-013 | terraform init+validate+plan live AWS (microservice) | live-aws | Verified |
|
||
| CAP-014 | terraform init+validate+plan live AWS (static-assets: CloudFront+WAF+S3) | live-aws | Verified |
|
||
| CAP-015 | DynamoDB outbox table exists + describable | live-aws | Verified |
|
||
| CAP-016 | S3 state bucket exists + readable | live-aws | Verified |
|
||
|
||
**IAM-gated cloud resources (6, escalated — not auto-verifiable):**
|
||
CAP-017..CAP-022 (DynamoDB contracts table, Lambda contract-ingestor, ECS
|
||
service live, CloudFront production stack, uptime-kuma, OIDC role). The
|
||
`acdl-spike-runner` IAM user lacks the permissions to verify these
|
||
(chicken-and-egg). The terraform plan path (CAP-013, CAP-014) proves the
|
||
code would deploy them; the local emulators (Phase 53) prove the runtime
|
||
behavior. Re-bootstrap of the OIDC role + IAM re-grant requires an admin
|
||
principal — escalated, not silently skipped. See
|
||
`CAPABILITY_INVENTORY-v1.10.md` §"Cloud capabilities NOT re-verified".
|
||
|
||
**Regression gate.** `bash scripts/run_regression.sh` re-runs all 16
|
||
auto-verifiable capabilities and fails closed on any non-Verified result.
|
||
|
||
## Constraints
|
||
|
||
- **Forge:** Gitea at `https://git.cloudinit.dev`, org `continuous-intelligence`.
|
||
- **CI runtime:** act_runner / Gitea Actions (reuses GitHub Actions workflow YAML).
|
||
- **Cloud:** AWS via OIDC federation. **Long-lived credentials are forbidden**
|
||
(§12.5). The v1.1 spike uses a temporary long-lived key **once** to bootstrap
|
||
OIDC (waiver D-034), then rotates it.
|
||
- **Engine:** Terraform adapter (the only adapter). L1/L2 are engine-agnostic
|
||
in shape; the adapter is the only engine-specific code.
|
||
- **State:** S3 (state files) + DynamoDB (locking), single-region in v1.
|
||
- **Environments:** dev (autonomous) → qa (QA HITL) → prod (SRE HITL) → dr
|
||
(SRE HITL). **Staging does not exist** (Path A locked).
|
||
- **Compute:** abstract / containerized / serverless. No VMs, bare metal, OS
|
||
lifecycle.
|
||
- **Autonomy:** Full. Escalation hooks: deploy, delete_data, merge_to_main.
|
||
|
||
## Anti-Goals (from `docs/vision.md` §7)
|
||
|
||
- Not an upstream development platform (no product backlogs, IDE, code authorship).
|
||
- Not a general-purpose AI (autonomy is narrow, bounded by policy envelopes).
|
||
- Not a legacy infrastructure bridge (no VMs/bare metal/OS).
|
||
- Not a permissive delivery highway (no escape hatches past confidence or HITL).
|
||
- Not a mutable audit log (VCS history ≠ regulatory evidence).
|
||
|
||
## Context
|
||
|
||
- The `acdl` repo exists at the org root. `acdl-contracts` and
|
||
`acdl-evidence` exist from the v1.0 demo and continue as the developer
|
||
surface and the audit-timeline host respectively.
|
||
- `docs/vision.md` and `docs/architecture.md` (v0.2) are the upstream
|
||
vision/architecture sources.
|
||
- The v1.0 demo (tag `v1.1.0`) is the reference of intent — it proved the
|
||
shape (L1/L2/contract/confidence/evidence/HITL) on stubs. v1.1+ replaces
|
||
the stubs with the real platform engine.
|
||
|
||
## Key Decisions (still load-bearing)
|
||
|
||
> The full decision history (D-001..D-213) is preserved across the v1.0–v1.24
|
||
> archive snapshot + the active-milestone CLARIFY/GRILL files. The decisions
|
||
> below remain load-bearing for v1.26 and are retained to avoid
|
||
> cross-file pointer-chasing on every read.
|
||
|
||
### v1.1 (D-034..D-047) — spike bootstrap, OIDC waiver, confidence inputs
|
||
|
||
| ID | Decision | Rationale | Outcome |
|
||
|----|----------|-----------|---------|
|
||
| D-034 | Temporary long-lived AWS key (waiver) used once in Phase 08 to bootstrap the state backend + IAM user; rotated/deactivated immediately after | §12.5 forbids long-lived creds; the bootstrap needed one `aws iam` call before the spike user + rotated key could take over | Spike achieves real `terraform plan` against AWS without violating the locked target after bootstrap. **CLOSED 2026-07-21: root key deactivated; the spike uses the rotated `acdl-spike-runner` key per D-039. Key ID redacted in v1.2 Phase 12 (P1-1).** |
|
||
| D-036 | Spike picks `l1-s3` + `l2-static-assets` | Simplest real AWS resource; smallest real `terraform plan`; proves the IR + adapter end-to-end | Spike scope fixed |
|
||
| D-039 | Spike-only waiver: per-run-rotated long-lived AWS key. OIDC federation deferred (Gitea Actions does NOT support `id-token: write` — go-gitea/gitea#36988). `scripts/rotate_spike_key.sh` rotates after each run. | §12.5 forbids long-lived creds; the waiver satisfies its *intent* (no *persistently* long-lived key). | Spike achieves real `terraform plan` without a persistently long-lived key; real OIDC is a v1.3+ deliverable. |
|
||
| D-040 | The 6 confidence-signal inputs are: policy (0.30), validation (0.25), freshness (0.10), source (0.15), history (0.10), nfrs (0.10). Weights frozen for v1. | Architecture §8 locks "six canonical inputs" but does not enumerate them. | Confidence signal has a concrete input enumeration. |
|
||
| D-042 | HITL approver identity in Gitea = `gitea.actor` of the `workflow_dispatch` run that sets `approve_{qa,prod,dr}=true`. Separation-of-duties reads `approver_qa` from the DynamoDB outbox and compares to the prod-dispatch `gitea.actor`. | Gitea has no Environments API; `gitea.actor` is the only approval-identity signal. | SoD design is concrete for the Gitea forge. |
|
||
| D-047 | v1.2 carries forward the D-039 per-run-rotated-key waiver. Real OIDC federation remains deferred (go-gitea/gitea#36988 still open). | §12.5 forbids long-lived creds; rotation hygiene satisfies the intent. | v1.2 achieves `terraform apply` without a persistently long-lived key. |
|
||
|
||
### v1.7 (D-048..D-060) — rename, production static-assets, contract ingestion, error reporting, module examples
|
||
|
||
| ID | Decision | Rationale | Outcome |
|
||
|----|----------|-----------|---------|
|
||
| D-049 | Production static-assets stack = S3 + CloudFront (OAC) + WAF. | Self-contained, domain-free production edge. Route53/ACM are domain-dependent and deferred. | Authors `cloudfront` + `waf` primitives. |
|
||
| D-050 | Deploy outputs: SSM Parameter Store (`SecureString`, KMS-encrypted, `/acdl/{env}/{contractId}/{output_name}`) + GitHub PR comment / job summary. | Two canonical mechanisms: SSM for runtime reads; PR comment for developers. No raw secrets in logs. | Implements `core/output_publisher.py` + two new pipeline stages. |
|
||
| D-051 | Contract ingestion storage = DynamoDB table `acdl-contracts` (PK `consumerRepo`, SK `contractId#submittedAt`, SSE via customer-managed CMK, PITR). | Enables historical queries, impact analysis, CMDB-style application-state queries. | Defines the table + Lambda. |
|
||
| D-053 | Kyverno adapter = K8s-native policy adapter translating `PolicyReport` results → `PolicyCheckResult`. Inactive for Terraform-only stacks. | The platform emits Terraform, not K8s manifests. The adapter activates when the GitOps reconciler emits K8s manifests. | Authors `adapters/kyverno/kyverno_adapter.py` + sample policies. |
|
||
| D-054 | Tagging standard = required-tag set (`acdl:owner`, `acdl:contract`, `acdl:environment`, `acdl:cost-center`) enforced by a Checkov custom YAML rule. | Closes the D-043 deferral (the SKIPPED `ACDL_TAG_NAMING` placeholder becomes a real check). | Authors `schemas/tagging-standard.json` + `adapters/terraform/policy/custom_rules/acdl_tagging.yaml`. |
|
||
| D-055 | Error reporting = the platform Lambda `report_error` action creates a GitHub issue on the platform repo (`acdl/acdl`). Gitea is excluded. | Unifies requirements around one mechanism. The Lambda holds a GitHub token (Secrets Manager) scoped to the platform repo. Idempotent (comments on existing open issue). | Implements the action + wires the `if: failure()` workflow step. |
|
||
| D-058 | Module examples = separate validated files in `modules/<name>/examples/` (`simple.yaml` + `complex.yaml` + variation files), validated against `schemas/contract.schema.json`. | Examples cannot drift from the schema silently. | Authors the example files; the platform-test pipeline validates them. |
|
||
| D-059 | Add an RDS primitive (`modules/l1/rds/`) with an `engine` input (enum: postgres, mysql, etc.) + a multi-engine example. | Concrete demonstration of multi-engine variation. | Authors the primitive + adapter expansion + examples. |
|
||
|
||
### v1.8 (D-061..D-072) — P1 remediation, encryption + deletion-protection by default, uptime, decommission
|
||
|
||
| ID | Decision | Rationale | Outcome |
|
||
|----|----------|-----------|---------|
|
||
| D-062 | SSM publisher fails loud (`RuntimeError`) when `ACDL_KMS_KEY_ID` unset. `ACDL_ALLOW_DEFAULT_KMS=1` escape hatch for local testing. | Silent AWS-managed-key use is the security gap; callers must set the env. | Implements fail-loud + escape hatch. |
|
||
| D-064 | Remove committed `terraform/spike/*.tf` entirely; adapter emits to per-run temp dir. | Cleaner; no stale fixtures. | Removes files + changes `run_platform.sh` target. |
|
||
| D-066 | Uptime deployment target: ECS Fargate (reuse existing `ecs-cluster` + `ecs-service` + `alb` primitives). | Most consistent with current platform; ALB gives a stable URL. | Authors uptime primitive on ECS Fargate. |
|
||
| D-068 | CMDB = DynamoDB `acdl-change-requests` table (PK `changeRequestId`, SK `submittedAt`). | Consistent with existing platform Lambda + DynamoDB pattern. | Adds the table + `validate_change_request` Lambda action. |
|
||
| D-069 | Encryption key granularity: per-stack CMK (one key per L2 deployment, tagged with `acdl:owner` + `acdl:environment`). | No shared keys across stacks; 90-day rotation at creation. | Authors `kms-key` primitive + L2 wiring. |
|
||
| D-070 | Decommission: new mode on the existing deploy pipeline (`mode: decommission`). 2-step with HITL SRE gates. | User chose existing pipeline with different behavior. | Adds decommission mode + HITL gates. |
|
||
| D-072 | Managed KMS fallback for standalone L1 deployments (no L2 CMK): adapter uses `alias/aws/<service>` with a stderr warning. `kms_key_arn` input is optional everywhere; `encryption_enabled` NFR defaults to true. | Requirement says "prioritize CMKs, fallback to managed KMS". | Implements fallback + warning. |
|
||
|
||
### Phase 07 open-decision resolutions (still load-bearing for the contract/confidence/HITL substrate)
|
||
|
||
| ID | Question | Resolution |
|
||
|---|---|---|
|
||
| W1.A | AI-refinement trigger | Joint condition: N ≥ 50 consecutive changes with zero rollbacks AND no L1/L2 incident in last 6 months AND Infra & Ops unilateral override. |
|
||
| W1.B | Multi-stack edge case rule | Permitted only for (a) DR-region mirror, (b) time-boxed experimental stack with TTL ≤ 30d, (c) explicit Infra & Ops approval with `multiStack.justification`. |
|
||
| W2.A | Tag mutability for prod | Tag for dev/qa, SHA for prod. Platform CLI resolves tag→SHA for prod-bound workflows. |
|
||
| BA.A | Initial L3B skill catalog | 5 skills: web API, worker, scheduled job, static asset, basic observability bootstrap. **Extended v1.18 (REQ-221/222):** 9 Atelier-derived production-grade engineering skills under `skills/` (api, security, data, testing, observability, errors, devops, infrastructure-as-code, compliance), indexed by `docs/skills.md`. |
|
||
| W3.D | L1/L2 standard versioning | Semver: interface → MAJOR, behavior → MINOR, lifecycle → PATCH. Pin model: L2 contracts pin L1 by `name@semver`; the resolver picks the highest compatible. MAJOR bumps require a new registry entry (immutable publication); old entry enters a 12-month deprecation window. |
|
||
| W3.E | Schema mandatory vs optional inputs | Per-env mandatory table: dev requires `stack` + `environment`; qa adds `validation.e2eSuite` + `validation.loadTest`; prod adds `runbook` + `dashboard` + `oncall`; dr adds `drDrillRef`. `inputs` map is always optional. `profile: agentic` fields optional everywhere. |
|
||
| BA.B | Confidence threshold tuning | Starting thresholds frozen for v1. Tuning begins in v1.2: track FP/FN per environment quarterly; override authority = Infra & Ops + SRE joint sign-off; any override is itself a confidence-event in the audit stream. |
|
||
| BA.C | On-call / operational ownership | Platform on-call = Infra & Ops rotation. Escalation: L3A/L3B halt → platform on-call pager (Sev2); consumer-visible outage → consumer on-call (Sev1) with platform on-call support. |
|
||
| BA.D | Cost / capacity governance | Cloud cost owner = Infra & Ops FinOps. Per-contract consumption reported monthly. Runaway spend: hard halt at 120% of contract-declared budget envelope via the confidence signal; override = FinOps + SRE joint sign-off. |
|
||
| BA.E | Consumer onboarding | Two paths: developer (L3A) — `getting-started` walks through contract schema + central pipeline template; citizen developer (L3B) — onboarding grants a scoped agent + skill catalog, no workflow authoring. Both end in a sandbox dev submission that must pass the confidence gate. |
|
||
| BA.F | Cross-platform evolution | The contract schema, IR, PolicyCheckResult, confidence signal, and audit stream are portable (engine- and forge-agnostic). Forge-specific code: workflow YAML, OIDC trust, CODEOWNERS, Environments. |
|
||
| Q1.3 | OpenTofu timing | **Deferred.** Not in v1 or v1.1. The engine abstraction (§12) makes OpenTofu a future adapter, not an architecture change. |
|
||
|
||
> **Prior milestone (v1.0 demo) decisions.** D-001..D-033 governed the
|
||
> stub-driven executive demo and remain valid **for the archived demo
|
||
> under `demo/`**. They are superseded by the v1.1+ decisions above for
|
||
> the real platform. Full text preserved in git history at tag `v1.1.0`
|
||
> and in `.ciagent/archive/PROJECT-v1.0-v1.24.md`.
|
||
|
||
> **v1.14–v1.24 decisions.** D-073..D-199 are preserved verbatim in
|
||
> `.ciagent/archive/PROJECT-v1.0-v1.24.md`. They remain valid for the
|
||
> milestones they governed; the active v1.26 milestone does not
|
||
> re-decide them.
|
||
|
||
> **v1.25 + v1.26 decisions.** D-200..D-213 (v1.26 CLARIFY) live in
|
||
> `.ciagent/CLARIFY.md`; v1.25 binding decisions live in the archive
|
||
> snapshot. Both sets are load-bearing for v1.26 execution.
|
||
|
||
## v1.25 — kyverno-json Unified Policy Engine (immediate predecessor, complete)
|
||
|
||
> Complete. Feature milestone — the primary compliance/policy tool becomes
|
||
> kyverno-json, implemented behind a swappable adapter. Tag `v1.24.5`
|
||
> (milestone release on the v1.24.x line). Branch `milestone/v1.25-kyverno-json`.
|
||
> **Full narrative preserved in `.ciagent/archive/PROJECT-v1.0-v1.24.md`.**
|
||
|
||
`kyverno-json` is a runtime from the Kyverno ecosystem that applies Kyverno
|
||
policies to **any JSON or YAML payload** — not just Kubernetes manifests.
|
||
v1.25 makes it the **primary tool of choice for compliance / policy
|
||
checks** in Nova, implemented as an **adapter** (the `PolicyEngine`
|
||
protocol) so the platform may one day replace it (e.g. OPA) without
|
||
touching the confidence signal or the pipeline.
|
||
|
||
### What v1.25 delivered (still load-bearing for v1.26)
|
||
|
||
- **Swappable `PolicyEngine` protocol** (`core/policy_engine.py`) — a
|
||
Python Protocol + registry selected from `config.json` (`policy.engine`,
|
||
default `"kyverno-json"`). `KyvernoJsonEngine` implements it (shells
|
||
to the `kyverno-json` CLI); a future `OpaEngine` implements the same
|
||
protocol. The confidence signal and pipeline never import the engine
|
||
directly — they go through the registry.
|
||
- **`KyvernoJsonEngine` adapter** (`adapters/kyverno-json/`) —
|
||
`evaluate(payload, policies) -> list[PolicyCheckResult]` translates
|
||
kyverno-json native output to the existing PCR schema. `is_configured()`
|
||
guard skips gracefully when the `kyverno-json` binary is absent (emits
|
||
`SKIPPED`, never breaks the pipeline).
|
||
- **Policies over all four Nova artifacts** under
|
||
`adapters/kyverno-json/policies/`: `contract/`, `stack-ir/`, `plan-json/`,
|
||
`meta/`.
|
||
- **`run_platform.sh` Step 5 wiring** — Checkov/Wiz still run and emit raw
|
||
PCRs; `KyvernoJsonEngine.evaluate()` runs plan-JSON policies in
|
||
parallel; both PCR lists merge into the confidence signal's `policy`
|
||
input. No change to `core/confidence_signal.py`.
|
||
- **Regression-gate-as-policy** — the capability checks in
|
||
`core/regression_verify.py` (CAP-013, CAP-023, CAP-024) became
|
||
declarative kyverno-json policies over the capability-inventory JSON
|
||
frontmatter.
|
||
- **`policy-engineer` persona** (custom, added in RESEARCH) — owns the
|
||
policy territory; declarative-policies constraint; kyverno-json +
|
||
JMESPath frameworks.
|
||
|
||
### v1.25 hard constraints (carried into v1.26)
|
||
|
||
- DO NOT change `schemas/policy_check_result.schema.json` shape in a way
|
||
that breaks existing adapters — the contract is the moat. The `engine`
|
||
enum already includes `"kyverno"` and `"opa"`; v1.25 records carry
|
||
`engine: "kyverno"` (no new enum value).
|
||
- DO NOT remove Checkov or Wiz adapters — they remain as raw-finding
|
||
sources feeding into kyverno-json meta-policies.
|
||
- DO NOT remove the `confidence_signal.py` `PENALTY["critical"]: None`
|
||
hard-override — it stays as defense-in-depth behind the declarative
|
||
`block-on-any-critical` meta-policy.
|
||
- DO NOT change `core/confidence_signal.py`'s input contract — it already
|
||
consumes `list[PolicyCheckResult]`; v1.25 only changed *who produces*
|
||
that list, not *what* the list is.
|
||
- The platform must function with `kyverno-json` absent — `is_configured()`
|
||
returns false → `SKIPPED` records → confidence signal proceeds.
|
||
|
||
### v1.25 Requirements
|
||
|
||
New requirements REQ-291..REQ-309 — full text in
|
||
`.ciagent/archive/REQUIREMENTS-v1.0-v1.24.md` §v1.25. Summary: engine
|
||
protocol + registry (REQ-291,292), kyverno-json engine impl (REQ-293,294),
|
||
contract policies (REQ-295,296), stack-IR policies (REQ-297,298,299),
|
||
plan-JSON policies + pipeline wiring (REQ-300,301,302), meta-policies
|
||
(REQ-303), regression-gate policies (REQ-304,305), docs + adapter README
|
||
(REQ-306,307), tests (REQ-308,309).
|
||
|
||
## v1.26 — Live Pilot Estate Activation (complete, tag `v1.25.5`, merged to main 2026-08-19)
|
||
|
||
> **Active milestone.** Feature milestone — the first real consumer estate
|
||
> (a stock exchange on a homegrown PoA blockchain, equities only) is
|
||
> activated against live AWS account `581513795199`, lifting D-096.
|
||
> Branch: `milestone/v1.26-pilot-activation`. Tags run on the **v1.25.x**
|
||
> patch line: `v1.25.0` (P0) → `v1.25.1..v1.25.4` (P1–P4) → `v1.25.5`
|
||
> (P5 final = milestone release).
|
||
>
|
||
> **Multi-project mode:** this milestone introduces a 2nd tracked project —
|
||
> `nova-blockchain-exchange` (Gitea repo
|
||
> `continuous-intelligence/nova-blockchain-exchange`, local clone
|
||
> `/root/nova-blockchain-exchange`). The platform repo (`acdl`) remains
|
||
> the platform source; the consumer repo owns the app code +
|
||
> `contract.yaml`. Both projects share the v1.26 milestone; the
|
||
> consumer's `.ciagent/` files live in `.ciagent/nova-blockchain-exchange/`.
|
||
|
||
### Why
|
||
|
||
NORTH_STAR.md has three Post-Pilot targets (Touchless Resolution ≥99%,
|
||
Human Escalation <0.1%, AI Decision Accuracy ≥99.5%) whose measurement
|
||
*pipeline* is grounded but whose *denominator* is zero — no consumer
|
||
estate has ever run. v1.25 shipped the swappable policy engine; v1.26
|
||
ships the first real consumer. The D-096 deferral (live AWS
|
||
re-provisioning) is the single blocker; the pre-run (Workstream A)
|
||
re-created the state bucket + outbox table, so the platform components
|
||
exist. The milestone grounds the metrics (outcome backfill +
|
||
escalation reason), wires the env JSON to the real account, and runs
|
||
the pilot end-to-end.
|
||
|
||
### What the milestone delivers
|
||
|
||
- **Homegrown PoA blockchain** (`nova-blockchain-exchange` repo) —
|
||
append-only blocks, single validator (pilot), deterministic block
|
||
production, T+1 settlement finality = block commit. Equities only.
|
||
- **Order-matching engine** — limit order book, price-time priority.
|
||
- **Settlement service** — T+1, idempotent, finality = block commit.
|
||
- **Consumer `contract.yaml`** — declares the exchange stack; validated
|
||
against `schemas/contract.schema.json`; per-env variants.
|
||
- **Consumer deploy via `deploy.yml@v1.25`** — the reusable workflow
|
||
applies the contract, runs the policy engine, computes the confidence
|
||
signal, gates qa/prod/dr with HITL attestation, and records every
|
||
decision in the Decision Ledger.
|
||
- **3 Post-Pilot metrics grounded** — outcome backfill (AI Decision
|
||
Accuracy), `reason='confidence'` escalation tag (Human Escalation
|
||
Frequency), and the pilot run itself (Touchless Resolution Rate
|
||
denominator activates).
|
||
- **3 kyverno-json policies extending v1.25** — settlement-finality
|
||
(securities-specific), pilot-readiness (no placeholder account), and
|
||
the existing meta-policies (block-on-any-critical, tagging-rules-agree)
|
||
apply over the pilot's PCRs.
|
||
- **Env-JSON `state_backend` wiring reconciliation** — the adapter reads
|
||
`state_backend.bucket` from the env JSON (closing the wiring gap); the
|
||
env JSONs are bound to account `581513795199`.
|
||
|
||
### v1.26 Requirements
|
||
|
||
New requirements REQ-310..REQ-322 — full text in
|
||
`.ciagent/REQUIREMENTS.md` §v1.26 + `.ciagent/nova-blockchain-exchange/REQUIREMENTS.md`.
|
||
Summary: blockchain core (REQ-310), order engine (REQ-311), settlement
|
||
(REQ-312), consumer contract (REQ-313), deploy invocation (REQ-314),
|
||
settlement-finality policy (REQ-315), pilot regression CAP (REQ-316),
|
||
outcome backfill (REQ-317), escalation reason (REQ-318), env-JSON wiring
|
||
(REQ-319), pilot-readiness policy (REQ-320), docs (REQ-321), DynamoDB L1
|
||
primitive (REQ-322 — the single platform-side module build-out; ECS + S3
|
||
already exist).
|
||
|
||
### v1.26 Hard constraints
|
||
|
||
- DO NOT lift D-083 (S3 Object Lock/JWS) — stays deferred; the SQLite
|
||
hash-chain + DynamoDB outbox is the pilot's audit record.
|
||
- DO NOT lift D-126 (hot path) — cold-only metrics are sufficient for
|
||
the pilot.
|
||
- DO NOT add multi-cloud (Azure/GCP) — Nova is AWS-only this milestone.
|
||
- DO NOT add ML forecasting — the Predictive/Reactive metric stays
|
||
deferred.
|
||
- DO NOT add bonds/derivatives/options — equities only (D-200).
|
||
- DO NOT add multi-validator BFT — single validator PoA (D-201).
|
||
- The consumer deploy MUST go through `deploy.yml@v1.25` — no direct
|
||
`terraform apply` bypassing the platform's gates.
|
||
|
||
### v1.26 phase status (shipped — tag `v1.25.5` = the v1.26 release, merged to main 2026-08-19)
|
||
|
||
- **P0** pre-execution (SPECIFY→CLARIFY→RESEARCH→IDEATE→PLAN→GRILL) — complete, tag `v1.25.0`.
|
||
- **P1** blockchain-core (REQ-310,311,312) — complete, tag `v1.25.1`.
|
||
- **P2** consumer-contract-and-deploy (REQ-313,314,322) — complete, tag `v1.25.2`.
|
||
- **P3** pilot-metrics-and-policies (REQ-315,316,317,318,319,320) — complete, tag `v1.25.3`.
|
||
- **P4** pilot-run-and-docs (REQ-316,321) — complete, tag `v1.25.4` (live apply against `581513795199` succeeded; confidence 0.800 pass; outcome backfilled).
|
||
- **P5** final review + audit + milestone ship — complete, tag `v1.25.5` = the v1.26 release (PROCEED; 0 P0 remain; audit CLEAN; merged to main).
|
||
|
||
> Phase-by-phase task breakdown, wave ordering, and persona assignments
|
||
> live in `.ciagent/PLAN.md` (the active phase plan, retained in full).
|
||
> v1.26 pre-execution artifacts (CLARIFY/GRILL/IDEATE/RESEARCH) are in
|
||
> git history (pre-v1.27-P0 commits); the v1.26 phase verifications +
|
||
> review are archived at `.ciagent/archive/{VERIFY-P03,VERIFY-P04,REVIEW-AUDIT-P05}.md`.
|
||
|
||
## v1.27 — PO State Catalog & Ciagent Compression (complete, tag `v1.26.3`, merged to main 2026-08-19)
|
||
|
||
> **NFR milestone — complete.** STATE.md authored (32 CAPs, 11 invariants,
|
||
> 10 domains). 8 outdated `.ciagent/` files archived (7 platform + 1
|
||
> consumer). PROJECT.md + ROADMAP.md v1.26 phase-status corrected.
|
||
> STATE.md wired into P-final ship discipline. Tags: `v1.26.0` (P0) →
|
||
> `v1.26.1..v1.26.2` (P1..P2) → `v1.26.3` (P3 final = milestone release).
|
||
> Review: 0 P0. Audit: reconstruction PASS, file/branch/commit discipline CLEAN.
|
||
> Full phase detail: `.ciagent/archive/` (v1.27 artifacts) + git history.
|
||
|
||
## v1.28 — CLI Canonicalization + Identity Layer (active)
|
||
|
||
> **Feature milestone — active.** The Nova CLI becomes installable from
|
||
> internal PyPI (CodeArtifact), every `core/` module is reachable as a
|
||
> `nova <subcommand>`, the CLI and Lambda functions share a single
|
||
> `core/` source tree, and Nova owns its identity layer end-to-end
|
||
> (sign-up through token vending) with no AWS-managed identity services
|
||
> in the path. Nova-idp is introduced: two Lambda functions (`nova-idp-auth`,
|
||
> `nova-idp-token-vend`), KMS-signed OIDC tokens, ABAC-gated token vending
|
||
> via the existing kyverno-json engine (INV-4 swappable), and PAT
|
||
> lifecycle (issuance, revocation, status).
|
||
>
|
||
> Tags run on the **v1.27.x** line: `v1.27.0` (P0) → `v1.27.1..v1.27.N`
|
||
> (execution phases) → `v1.27.(N+1)` (final phase = milestone release).
|
||
> Milestone branch: `milestone/v1.28-cli-identity`.
|
||
|
||
### v1.28 ID allocations (re-mapped — no collisions with shipped history)
|
||
|
||
- **Decisions:** `D-226..D-231` (authored in CLARIFY). Repo decision
|
||
namespace is `D-NNN` (max D-225); no `D-NEW-*` namespace exists.
|
||
- **Requirements:** `REQ-323..REQ-353` (31 REQs, mapping the spec's
|
||
REQ-001..REQ-031 1:1). Max existing REQ = REQ-322.
|
||
- **Capabilities:** `CAP-033..CAP-038` (mapping the spec's CAP-025..CAP-030).
|
||
Existing CAP-025..032 are blockchain/pilot — collision avoided.
|
||
- **Invariants:** `INV-12..INV-17` (mapping the spec's INV-63,64,65,18..21,34).
|
||
Max existing INV = INV-11.
|
||
- **`kj` engine → kyverno-json.** The spec references a `kj` engine; the
|
||
repo's actual policy engine is `kyverno-json` (INV-4 swappable). v1.28
|
||
uses kyverno-json as the ABAC evaluator for token-vend; no new `kj`
|
||
engine is built. This is a CLARIFY-grounded re-mapping, not a silent
|
||
assumption (D-229).
|
||
|
||
### v1.28 Requirements
|
||
|
||
New requirements REQ-323..REQ-353 — full text in
|
||
`.ciagent/REQUIREMENTS.md` §v1.28. Summary by priority:
|
||
|
||
- **P1 — CLI Substrate (REQ-323..REQ-328):** CodeArtifact wheel + Lambda
|
||
layer pipeline; CLI subcommand per `core/` module; `nova init`
|
||
scaffolding; `nova cli-action` published to GitHub + Gitea;
|
||
`mode_resolver.py` (flag → env → credential type → TTY); audit
|
||
emission with `mode` + `selection_reason`.
|
||
- **P2 — Lambda Packaging + Identity Layer (REQ-329..REQ-344):** dual-use
|
||
`core/lambda/contract_ingestor.py`; local env synthesizer; JWS signing
|
||
key from PAT; `nova-idp-auth` Lambda (Argon2id, DynamoDB); DynamoDB
|
||
tables (`nova-users`, `nova-sessions`, `nova-password-resets`);
|
||
`nova-idp-token-vend` Lambda (KMS-signed OIDC, JWKS endpoint); kyverno-json
|
||
ABAC policy at `platform/abac/token-vend.policy`; `nova idp setup`
|
||
(`--check/--apply/--verify`); CloudFormation review; PAT issuance +
|
||
hashes in DynamoDB; `nova auth login/revoke/status`.
|
||
- **P3 — Documentation (REQ-345..REQ-347):** operator guide for
|
||
`nova idp setup`; developer guide for `nova auth login`; identity-layer
|
||
threat model.
|
||
- **P4 — Integration Testing (REQ-348..REQ-351):** E2E sign-up → sign-in →
|
||
token-vend → apply → audit; property tests for `mode_resolver`; KMS
|
||
round-trip test; PAT revocation SLO test (≤60s P95).
|
||
- **P5 — Capability Gate (REQ-352..REQ-353):** CAP-033..038 verification
|
||
gates wired into CI.
|
||
|
||
### v1.28 Hard constraints
|
||
|
||
- DO NOT depend on Cognito, IAM Identity Center, or any AWS-managed
|
||
identity service for sign-up/sign-in/token-vending (NFR-5). Nova-idp
|
||
signs OIDC tokens directly via KMS. (Note: no Cognito exists in the
|
||
repo today — this is a greenfield build, not a "Cognito drop".)
|
||
- DO NOT build a new `kj` engine — use kyverno-json (INV-4).
|
||
- DO NOT enforce MFA/TOTP for prod/dr this milestone — ship the code path,
|
||
enforce in v1.21+ (deferred, INV scope).
|
||
- DO NOT add WebAuthn/FIDO2, upstream IdP federation, or password breach
|
||
detection — deferred to v1.23+.
|
||
- DO NOT add Lambda layer auto-update on `core/` changes — v1.18 ships
|
||
manual `nova layer update`; v1.19 adds CI-triggered auto-update.
|
||
- The token-vend Lambda MUST evaluate the kyverno-json ABAC policy before
|
||
signing; allow/deny decisions MUST be emitted to the audit stream
|
||
(NFR-9, D-227).
|
||
- `nova idp setup --apply` MUST present the CloudFormation template for
|
||
review before any resource is created (NFR-10).
|
||
|
||
### v1.28 phase status (active — phase 0 in progress)
|
||
|
||
- **P0** pre-execution (SPECIFY→CLARIFY→RESEARCH→PLAN→GRILL→MVP/UX) — in
|
||
progress, target tag `v1.27.0`.
|
||
- **P1..PN** execution phases — planned in PLAN.md.
|
||
- **P(N+1)** final review + audit + milestone ship — target tag
|
||
`v1.27.(N+1)` = the v1.28 release.
|
||
|
||
> Phase-by-phase task breakdown, wave ordering, and persona assignments
|
||
> will live in `.ciagent/PLAN.md`. Authoritative resume state:
|
||
> `.ciagent/CHECKPOINT.json`. |