Files
praxis/.ciagent/PLAN.md
T
Praxis CI 813bd586d6 docs(milestone): merge v0.3-mastery-scoring → main
v0.3 milestone merged to main. Mastery scoring + competency rubrics +
verifiable credentials (formative-tier) shipped. 13/13 REQ-IDs covered.
Next milestone: v0.4 (operator tier — cohort dashboard + auth + Postgres).

---ci---
project: praxis
phase: 2
milestone: v0.3
status: complete
milestone_complete: true
milestone_merged_to_main: true
---/ci---
2026-08-04 00:14:59 +00:00

31 KiB
Raw Blame History

Praxis — v0.3 Execution Plan (Mastery Scoring + Competency Rubrics + VC Issuance)

Milestone: v0.3 (Mastery scoring + competency rubrics + verifiable credentials) Phases: 1 execution phase (P1: mastery core + IRT + scenarios + paths + VC issuance) + final phase (P2: review + ship) Ship: v0.1.3 (Phase 0) → v0.1.4 (P1) → v0.1.5 (P2 = v0.3 milestone release) Status: plan (grill-amended — operator tier deferred to v0.4 per GRILL-v0.3.md Axis 2 + Axis 8) Autonomy: full Parallelization: enabled, max 5 concurrent agents Personas active: lead-developer, backend-engineer, data-engineer, security-engineer (frontend-engineer + devops-engineer DEACTIVATED — no UI, no new deploy scripts in v0.3) Date: 2026-08-03


Grill Amendments (binding — per GRILL-v0.3.md)

The grill (GO-WITH-CONDITIONS, 4 MUST) restructured this plan:

  1. Axis 2 (MUST) — Split the milestone. The operator tier (REQ-DASH-01, REQ-AUTH-01, REQ-MT-01/02 + associated NFRs) is deferred to v0.4. v0.3 is now a clean learner-facing mastery milestone. This restores the original ROADMAP intent (dashboard was v0.8) and avoids the hybrid SQLite+Postgres topology in v0.3.
  2. Axis 8 (MUST) — VC issuance moves to P1. VC issuance is a learner-facing consequence of mastery (D-048), not an operator feature. Issuer keys are SQLite-backed in v0.3 (Postgres takes over in v0.4 when the operator tier arrives).
  3. Axis 3 (MUST) — VC interop + key-rotation tests added. TASK-12-07 (external W3C verifier interop) + TASK-12-08 (key-rotation operational drill).
  4. Axis 4 (MUST) — Three technical-risk fixes. (a) VC labeled formative in payload + verification + REQ-MAST-03. (b) R-AUTH-01 deferred to v0.4 with the operator surface (no auth in v0.3 → no cookie issue). (c) Evidence-extraction fallback changed from silent-fail-to-zero to scoring_inconclusive with learner-visible retry signal.

FIX conditions (non-blocking, tracked in VERIFY): re-task SLICE-12/13 (now moot for v0.3 — operator tier deferred), wire VC trigger (resolved — VC now in P1), Postgres-failure semantics (deferred to v0.4), real-LLM smoke test (added to P1 SLICE-08), k-anonymity differencing-attack test (deferred to v0.4), reconciliation drift-correction test (deferred to v0.4), de-escalation weight clarification (static in v0.3 — dynamic re-weighting is a future feature).


Phase Split Rationale (post-grill)

v0.3 is now a single execution phase (P1) + final review/ship (P2):

  • P1 (Mastery Core + VC Issuance): rubric engine, IRT, scenario library (≥6 CS scenarios), path engine (6-week), mastery score + gate logic, VC issuer (W3C VC 2.0, Ed25519, SQLite-backed issuer keys, public verification endpoint). All learner-facing. Shippable as v0.1.4.
  • P2 (Final): review + audit + milestone ship (v0.1.5 = v0.3 milestone release).

The operator tier (cohort dashboard, auth, Postgres) is v0.4 — a separate milestone with its own phase 0. This keeps v0.3 honest: one milestone, one shippable learner-facing deliverable, no hybrid storage, no operator auth surface.


Deferred to v0.4 (operator tier — per grill Axis 2)

The following REQ-IDs are deferred to v0.4 and removed from v0.3 scope:

  • REQ-DASH-01 (cohort dashboard) — was v0.8 on original ROADMAP; v0.4 is still ahead of that but follows the grill's "split the milestone" verdict
  • REQ-AUTH-01 (operator auth) — no operator surface in v0.3 → no auth needed
  • REQ-MT-01, REQ-MT-02 (operator Postgres, cohort aggregation) — no operator tier in v0.3
  • REQ-NFR-DASH-01, REQ-NFR-DASH-02, REQ-NFR-AUTH-01, REQ-NFR-MT-01 — associated NFRs

v0.3 REQ-IDs (post-grill): 13 (REQ-MAST-01/02/03, REQ-SCEN-02/03/04, REQ-PATH-02 + 6 NFRs: REQ-NFR-MAST-01/02, REQ-NFR-VC-01/02, REQ-NFR-IRT-01). REQ-MAST-04 is a principle (accepted).


Phase 1 — Mastery Core (learner-facing mastery layer)

Branch: phase/01-mastery-core → merged to milestone/v0.3-mastery-scoring Ship: v0.1.4 (patch release, feature milestone type) REQ-IDs covered: REQ-MAST-01, REQ-MAST-02, REQ-SCEN-02, REQ-SCEN-03, REQ-SCEN-04, REQ-PATH-02, REQ-NFR-MAST-01, REQ-NFR-MAST-02, REQ-NFR-IRT-01 Slices: 8 vertical slices in 4 waves Total tasks: 38

Wave Slices Parallel slots Description
1 SLICE-01, SLICE-02 2 Rubric schema + scenario library schema (parallel — disjoint file territories)
2 SLICE-03, SLICE-04, SLICE-05 3 Rubric scoring engine + IRT engine + path engine (parallel — all depend on W1 schemas, disjoint modules)
3 SLICE-06, SLICE-07 2 Scenario library content (≥6 CS scenarios) + mastery score + gate logic (parallel — SLICE-06 authors scenarios, SLICE-07 wires scoring into session_recorder)
4 SLICE-08 1 Integration tests + mastery-gate audit log + real-LLM smoke test (depends on all prior)
5 SLICE-09 1 VC issuer + verification endpoint + interop/rotation tests (depends on SLICE-07 gate-open trigger)

Wave dependency graph

Wave 1 ────────────────────────────────────────
  SLICE-01 (rubric YAML schema + loader)
  SLICE-02 (scenario library schema + index + loader)
        │
        ▼
Wave 2 ────────────────────────────────────────
  SLICE-03 (rubric scoring engine: evidence extractor + rule scorer)  ← depends on SLICE-01
  SLICE-04 (IRT engine + theta persistence)                          ← depends on SLICE-02 (scenario difficulty)
  SLICE-05 (path engine: 6-week structure + progression)             ← depends on SLICE-02 (scenario library)
        │
        ▼
Wave 3 ────────────────────────────────────────
  SLICE-06 (≥6 expert CS scenarios + index.yaml + rubric mapping)    ← depends on SLICE-01, SLICE-02
  SLICE-07 (mastery score + gate logic + session_recorder hooks)     ← depends on SLICE-03, SLICE-04, SLICE-05
        │
        ▼
Wave 4 ────────────────────────────────────────
  SLICE-08 (integration tests + mastery-gate audit log in SQLite + real-LLM smoke)  ← depends on all prior
        │
        ▼
Wave 5 ────────────────────────────────────────
  SLICE-09 (VC issuer: Ed25519 + JCS + Status List + verification + interop + rotation)  ← depends on SLICE-07 (gate-open trigger)

Persona load distribution (P1)

Persona Tasks Primary territory
backend-engineer 20 server/mastery/**, server/scenarios/library.py, server/paths/**, server/session_recorder.py extension
security-engineer 8 server/vc/** (Ed25519 issuer, JCS, Status List, verification endpoint, interop + rotation tests)
data-engineer 6 db/migrations/0003_mastery.sql (learner_ability, mastery_progress, issuer_keys, issued_credentials, mastery_gate_events tables), db/store.py v0.3 additions
lead-developer 6 pyproject.toml deps, integration test orchestration, cross-persona coordination
frontend-engineer 0 DEACTIVATED (no UI in v0.3 — dashboard is v0.4)
devops-engineer 0 DEACTIVATED (no new deploy scripts)

Total P1 tasks: 40 (was 38 + 8 VC - 6 rebalanced; +2 grill interop/rotation tests)


SLICE-01: Rubric Schema + Loader (W1)

  • Goal: Define the competency rubric YAML format + Pydantic model + loader so scenarios can reference rubric criteria.
  • REQ-IDs covered: REQ-MAST-01 (partial — schema only), REQ-NFR-MAST-01 (determinism foundation)
  • Wave: 1
  • Dependencies: none
  • Persona: data-engineer (schema), backend-engineer (loader)

Tasks

TASK-01-01 — Rubric YAML schema definition

  • Persona: data-engineer
  • File: rubrics/customer_service.yaml (new — refund/complaint archetype per RESEARCH §6.2)
  • Content: 4 criteria (empathy 0.35, resolution 0.30, de-escalation 0.20, professionalism 0.15), 5-level anchors each (level 1=fail … 5=mastery/entrustable, per RESEARCH §2), per-archetype weights (D-039 amendment). Professionalism = conjunctive floor ≥2.

TASK-01-02 — Rubric Pydantic model

  • Persona: backend-engineer
  • File: server/mastery/rubric_schema.py (new)
  • Content: Rubric, RubricCriterion, RubricLevel models. Fields: id, skill, criteria[{id, name, weight, levels[{level, anchor, signals[]}]}]. Validate weights sum to 1.0. Validate 5 levels per criterion.

TASK-01-03 — Rubric loader

  • Persona: backend-engineer
  • File: server/mastery/rubric_loader.py (new)
  • Content: load_rubric(skill: str) -> Rubric — loads rubrics/<skill>.yaml, parses via Pydantic. Caches in-memory. Validates against schema.

TASK-01-04 — Rubric unit tests

  • Persona: backend-engineer
  • File: tests/test_rubric_schema.py (new)
  • Content: load valid rubric, reject invalid weights, reject missing levels, criterion lookup by id, weight sum validation.

SLICE-02: Scenario Library Schema + Index + Loader (W1)

  • Goal: Extend the v0.1 scenario schema (D-018) with rubric mapping + library index manifest + loader for multi-scenario selection.
  • REQ-IDs covered: REQ-SCEN-03 (partial — schema), REQ-SCEN-04 (partial — format extension)
  • Wave: 1
  • Dependencies: none (parallel with SLICE-01 — disjoint files)
  • Persona: backend-engineer

Tasks

TASK-02-01 — Extend Scenario schema with rubric mapping + IRT fields

  • Persona: backend-engineer
  • File: server/scenarios/schema.py (extend existing)
  • Content: Add rubric_criteria: list[{criterion_id, weight, evidence_required}] field to Scenario. Add irt_target_p: float = 0.7 field (D-035 practice default). Add version: str (semver, D-036). Add generated_from: str | None (AI-variation backref, D-036). Add intent_hash: str | None (structural drift detection). Keep backward compat with v0.1 scenario YAML.

TASK-02-02 — Scenario index manifest

  • Persona: backend-engineer
  • File: scenarios/index.yaml (new — slim manifest per RESEARCH §D)
  • Content: list of {id, path, title, difficulty, failure_mode, rubric_criteria, version, author, generated_from}. ~50 lines/scenario metadata. Updated when scenarios are added.

TASK-02-03 — Scenario library loader

  • Persona: backend-engineer
  • File: server/scenarios/library.py (new)
  • Content: ScenarioLibrary class — loads scenarios/index.yaml, loads individual scenario YAMLs on demand, validates against schema. list_by_path(path), list_by_difficulty(range), get(scenario_id), select_for_theta(theta, path) (IRT-aware selection targeting ~50% or ~70% per irt_target_p). Enforces MIN_COVERAGE = 2 scenarios per rubric criterion (CI check, RESEARCH §D).

TASK-02-04 — Library unit tests

  • Persona: backend-engineer
  • File: tests/test_scenario_library.py (new)
  • Content: load index, list by path, select_for_theta, MIN_COVERAGE validation, reject invalid semver, AI-variation backref validation.

SLICE-03: Rubric Scoring Engine (W2)

  • Goal: Implement the deterministic rubric scoring flow: LLM-extracts-evidence, rules-score-evidence (D-038, REQ-NFR-MAST-01).
  • REQ-IDs covered: REQ-MAST-01 (scoring logic), REQ-NFR-MAST-01 (determinism)
  • Wave: 2
  • Dependencies: SLICE-01 (rubric schema)
  • Persona: backend-engineer

Tasks

TASK-03-01 — Evidence extractor (LLM, off-voice-path)

  • Persona: backend-engineer
  • File: server/mastery/evidence_extractor.py (new)
  • Content: async extract_evidence(turns, rubric_criteria) -> list[Evidence]. Calls deepseek-v4-flash:cloud, temp=0, JSON-schema-validated output: [{criterion_id, quote, signals: [...]}]. Critical: fuzzy-match quote against transcript (rapidfuzz or difflib) → reject + re-extract on mismatch (R-MAST-02). Max 2 re-extraction attempts; on final failure, mark scenario as scoring_inconclusive — do NOT count toward gate, do NOT penalize learner, surface 'technical issue, please retry' in the debrief (grill Axis 4 MUST #3 — silent fail-to-zero is unacceptable). Log the failure for operator review.

TASK-03-02 — Rule-based scorer (deterministic)

  • Persona: backend-engineer
  • File: server/mastery/rubric_scorer.py (new)
  • Content: score(evidence, rubric) -> list[CriterionScore]. Maps signals → 1-5 level per criterion via rubric YAML level anchors (each level has a signals[] list — match evidence signals to level signals). Deterministic — no LLM. Output: [{criterion_id, level, weight, evidence_quote}].

TASK-03-03 — Mastery Score computation (deterministic)

  • Persona: backend-engineer
  • File: server/mastery/mastery_score.py (new)
  • Content: compute_scenario_score(criterion_scores, rubric) -> ScenarioScore (weighted mean + conjunctive floor: every criterion ≥2, scenario mean ≥3.0 to pass). compute_path_score(passing_scenario_scores) -> PathScore (mean over passing scenarios only). check_gate(path_score, distinct_passed_count) -> bool (≥3 distinct passed AND ≥3.5 — D-032).

TASK-03-04 — Scoring unit tests

  • Persona: backend-engineer
  • File: tests/test_rubric_scoring.py (new)
  • Content: evidence extraction with mocked LLM, quote fuzzy-match rejection, rule-based scoring determinism (same input → same output), conjunctive floor enforcement, gate logic.

TASK-03-05 — Evidence extractor integration test (mocked LLM)

  • Persona: backend-engineer
  • File: tests/test_evidence_extractor_integration.py (new)
  • Content: end-to-end extraction → scoring with a mocked LLM returning canned evidence. Verify JSON schema validation, quote matching, deterministic scoring.

SLICE-04: IRT Engine + Theta Persistence (W2)

  • Goal: Implement 1PL/Rasch IRT with Bayesian theta update, persisted to SQLite (D-046, REQ-NFR-IRT-01).
  • REQ-IDs covered: REQ-SCEN-02, REQ-NFR-IRT-01
  • Wave: 2
  • Dependencies: SLICE-02 (scenario difficulty field)
  • Persona: backend-engineer (engine), data-engineer (SQLite table)

Tasks

TASK-04-01 — IRT engine

  • Persona: backend-engineer
  • File: server/mastery/irt.py (new)
  • Content: class IRTEngine: P_success(theta, b) -> float (logistic(θ−b)). update_theta(theta, sigma_sq, outcome, b) -> (new_theta, new_sigma_sq) (Gaussian-approximation Bayesian: θ ← θ + (outcome P) × σ²/(σ² + 1); σ² shrinks per observation). select_scenario(theta, library, path, target_p) -> Scenario (picks scenario with b closest to θ logit(target_p)). Cold-start: θ=0, σ²=1; fall back to scenario.difficulty until ≥5 observations (R-IRT-01).

TASK-04-02 — Theta persistence (SQLite)

  • Persona: data-engineer
  • File: db/migrations/0003_mastery.sql (new — adds learner_ability + mastery_progress tables), db/store.py (extend)
  • Content: learner_ability table (learner_id, path, theta REAL, sigma_sq REAL, observations INTEGER, updated_at). mastery_progress table (learner_id, path, current_week INTEGER, scenarios_passed_json TEXT, mastery_score REAL, gate_open bool, updated_at). PraxisStore.get_ability(), set_ability(), get_progress(), set_progress() async methods.

TASK-04-03 — IRT unit tests

  • Persona: backend-engineer
  • File: tests/test_irt.py (new)
  • Content: P_success correctness, theta update convergence, cold-start fallback, select_scenario targeting, sigma_sq shrinkage.

TASK-04-04 — Theta persistence integration test

  • Persona: data-engineer
  • File: tests/test_learner_ability_db.py (new)
  • Content: get/set ability round-trip, get/set progress round-trip, migration idempotency, concurrent writes (aiosqlite).

SLICE-05: Path Engine (W2)

  • Goal: Implement the 6-week path structure with mastery gates (D-037, REQ-PATH-02).
  • REQ-IDs covered: REQ-PATH-02
  • Wave: 2
  • Dependencies: SLICE-02 (scenario library — paths reference scenarios)
  • Persona: backend-engineer

Tasks

TASK-05-01 — Path YAML schema + Pydantic model

  • Persona: backend-engineer
  • File: server/paths/schema.py (new)
  • Content: Path model: slug, name, skill, weeks[{week, title, scenario_ids[], gate: {required_scenarios: int, required_score: float}}]. Validate 6 weeks. Validate scenario_ids exist in library.

TASK-05-02 — Customer Service path YAML

  • Persona: backend-engineer
  • File: paths/customer_service.yaml (new)
  • Content: 6 weeks per PRD §6.4. Week 1: basics (refund scenario). Week 2: escalation. Week 3: policy exceptions. Week 4: multi-issue. Week 5: recovery. Week 6: mastery demonstration. Each week references ≥1 scenario from the library (SLICE-06). Gate: ≥3 distinct scenarios passed, score ≥3.5 (D-032).

TASK-05-03 — Path engine (progression logic)

  • Persona: backend-engineer
  • File: server/paths/engine.py (new)
  • Content: PathEngine: load_path(slug) -> Path. current_week(progress) -> int. check_gate(progress, week) -> bool (delegates to mastery_score.check_gate). advance_week(progress) -> progress (D-048). is_path_complete(progress) -> bool (week 6 gate open).

TASK-05-04 — Path unit tests

  • Persona: backend-engineer
  • File: tests/test_path_engine.py (new)
  • Content: load path, validate 6 weeks, gate check, week advancement, path completion.

SLICE-06: Scenario Library Content (W3)

  • Goal: Author ≥6 expert Customer Service scenarios filling the 6-week path (D-047, REQ-SCEN-03).
  • REQ-IDs covered: REQ-SCEN-03, REQ-SCEN-04 (expert-authored; AI variations in P2 or later)
  • Wave: 3
  • Dependencies: SLICE-01 (rubric), SLICE-02 (library schema)
  • Persona: lead-developer (content authoring — domain expertise), backend-engineer (validation)

Tasks

TASK-06-01 — Author 6 CS scenarios

  • Persona: lead-developer
  • Files: scenarios/customer_service/cs_refund_ca_v01.yaml (exists — extend with rubric mapping), scenarios/customer_service/cs_escalation_ca_v02.yaml (new), scenarios/customer_service/cs_policy_exception_ca_v03.yaml (new), scenarios/customer_service/cs_multi_issue_ca_v04.yaml (new), scenarios/customer_service/cs_recovery_ca_v05.yaml (new), scenarios/customer_service/cs_mastery_demonstration_ca_v06.yaml (new)
  • Content: Each scenario: extends v0.1 schema with rubric_criteria (mapped to the 4 CS criteria), irt_target_p (0.7 for practice weeks, 0.5 for mastery-demonstration week 6), version: 1.0.0, author: expert. Difficulty 1-5 across weeks. Failure modes vary (escalates_unresolved, policy_rigid, multi_issue_drop, recovery_missed).

TASK-06-02 — Update index.yaml manifest

  • Persona: lead-developer
  • File: scenarios/index.yaml (update)
  • Content: All 6 scenarios listed with metadata. MIN_COVERAGE = 2 per criterion verified (each of empathy/resolution/de-escalation/professionalism exercised by ≥2 scenarios).

TASK-06-03 — Scenario validation tests

  • Persona: backend-engineer
  • File: tests/test_scenario_library_content.py (new)
  • Content: all 6 scenarios load via schema, rubric_criteria reference valid criterion IDs, MIN_COVERAGE per criterion, semver valid, index.yaml in sync with files.

SLICE-07: Mastery Score + Gate Logic + Session Recorder Hooks (W3)

  • Goal: Wire the rubric scoring + IRT + path progression into the session end flow (server/session_recorder.py).
  • REQ-IDs covered: REQ-MAST-02, REQ-NFR-MAST-02 (auditability — SQLite log)
  • Wave: 3
  • Dependencies: SLICE-03 (scoring), SLICE-04 (IRT), SLICE-05 (path)
  • Persona: backend-engineer

Tasks

TASK-07-01 — Extend session_recorder.py with mastery hooks

  • Persona: backend-engineer
  • File: server/session_recorder.py (extend existing)
  • Content: After existing end() logic: (1) call evidence_extractor.extract_evidence(turns, scenario.rubric_criteria), (2) rubric_scorer.score(evidence, rubric), (3) mastery_score.compute_scenario_score(...), (4) irt.update_theta(...), (5) path_engine.check_gate + advance_week, (6) record mastery_gate_event in SQLite mastery_gate_events table (REQ-NFR-MAST-02 audit), (7) if week-final gate open → call vc_issuer.issue_credential(...) (SLICE-09) — VC issuance is wired here, not in a later phase (grill Axis 8 MUST). All off the voice path (async, after session end). If evidence extraction returns scoring_inconclusive, skip steps 2-7 and surface retry in debrief.

TASK-07-02 — Mastery gate event SQLite table

  • Persona: data-engineer
  • File: db/migrations/0003_mastery.sql (extend), db/store.py (extend)
  • Content: mastery_gate_events table (id, learner_id, path, week, scenarios_passed_json, rubric_scores_json, mastery_score, gate_opened_at). PraxisStore.record_gate_event() async method.

TASK-07-03 — Mastery integration test (end-to-end scoring flow)

  • Persona: backend-engineer
  • File: tests/test_mastery_integration.py (new)
  • Content: simulate a session with turns → run mastery flow → verify scenario score, theta update, progress advancement, gate event recorded. Mocked LLM for evidence extraction. Verify determinism (same input → same scores).

TASK-07-04 — IRT selection integration (next-scenario recommendation)

  • Persona: backend-engineer
  • File: server/scenarios/library.py (extend), tests/test_irt_selection_integration.py (new)
  • Content: library.select_for_theta(theta, path) picks the next scenario. Integration test: given a theta and a path, verify the selected scenario targets the right P.

SLICE-08: Integration Tests + Mastery-Gate Audit Log (W4)

  • Goal: End-to-end P1 integration tests + verify the mastery-gate audit log is complete and queryable.
  • REQ-IDs covered: REQ-NFR-MAST-02 (full auditability)
  • Wave: 4
  • Dependencies: all prior slices
  • Persona: lead-developer (orchestration), backend-engineer (tests)

Tasks

TASK-08-01 — End-to-end P1 smoke test

  • Persona: lead-developer
  • File: scripts/test_mastery_e2e.py (new)
  • Content: simulate 3 sessions across 3 distinct scenarios → verify mastery gate opens after 3 passing scenarios with score ≥3.5. Verify theta converges. Verify progress advances. Verify gate events recorded.

TASK-08-02 — Audit log queryability test

  • Persona: backend-engineer
  • File: tests/test_gate_audit_log.py (new)
  • Content: query mastery_gate_events by learner, by path, by date range. Verify evidence (scenarios_passed, rubric_scores) is persisted and reconstructable.

TASK-08-03 — P1 verification matrix

  • Persona: lead-developer
  • File: .ciagent/VERIFY-P1.md (new — pre-verify checklist for the verify stage)
  • Content: REQ-ID → test mapping. Confirm all P1 REQ-IDs have covering tests.

TASK-08-04 — Real-LLM evidence extraction smoke test (grill Axis 7 FIX #1)

  • Persona: backend-engineer
  • File: scripts/test_real_llm_evidence.py (new — staging-gated, requires OLLAMA_API_KEY)
  • Content: run one real session transcript through the actual deepseek-v4-flash:cloud evidence extractor. Verify output is valid JSON with fuzzy-matching quotes. This runs only in staging (gated by PRAXIS_RUN_REAL_LLM_TESTS=1 env). Mocked-LLM tests stay in CI. Validates that the extraction prompt works, not just the scoring logic.

SLICE-09: VC Issuer + Verification Endpoint + Interop/Rotation Tests (W5)

  • Goal: Implement Ed25519-signed W3C VC 2.0 issuance + public verification + Status List revocation, SQLite-backed issuer keys (D-033, D-042, D-043, REQ-MAST-03, REQ-NFR-VC-01, REQ-NFR-VC-02). VC labeled formative per grill Axis 4 MUST #1.
  • REQ-IDs covered: REQ-MAST-03, REQ-NFR-VC-01, REQ-NFR-VC-02
  • Wave: 5
  • Dependencies: SLICE-07 (gate-open trigger — TASK-07-01 step 7 calls issue_credential)
  • Persona: security-engineer (issuer + crypto), data-engineer (SQLite issuer_keys/issued_credentials tables)

Tasks

TASK-09-01 — SQLite issuer keys + issued_credentials tables

  • Persona: data-engineer
  • File: db/migrations/0003_mastery.sql (extend), db/store.py (extend)
  • Content: issuer_keys table (id, public_key TEXT, private_key_enc BLOB, status TEXT active|superseded, created_at). issued_credentials table (id, learner_id, vc_payload_json, signature_b64, status active|revoked, issued_at). PraxisStore async methods: init_issuer_key(), get_active_signing_key(), get_public_key(key_id), insert_credential(), get_credential(), set_credential_status(). Private key encrypted at rest with PRAXIS_VC_ISSUER_KEY root key from env (D-042).

TASK-09-02 — Ed25519 issuer key management + VC payload builder + JCS + signing

  • Persona: security-engineer
  • File: server/vc/issuer_keys.py (new), server/vc/issuer.py (new)
  • Content: init_issuer_key(store, root_key) -> KeyPair — generate Ed25519 (pynacl), encrypt private key, store in SQLite. build_vc_payload(learner_ref, path, scenarios_passed, rubric_score, completed_weeks, evidence) -> dict (W3C VC 2.0: scenariosPassed, rubricScore, completedWeeks: 6, evidence, issuedAt, validUntil: +3y, credentialTier: "formative" per grill Axis 4). canonicalize(payload) -> bytes (JCS via canonicaljson). sign(payload, signing_key) -> str (eddsa-jcs-2022). issue_credential(...) -> str (stores in SQLite).

TASK-09-03 — Bitstring Status List (revocation)

  • Persona: security-engineer
  • File: server/vc/status_list.py (new)
  • Content: BitstringStatusList — one bitstring per status list, indexed by credential sequence. set_status(credential_idx, revoked), get_status(credential_idx) -> bool. Persisted in SQLite (status_lists table or adjacent to issuer_keys). Revocation latency = next verify call (status list fetched from SQLite on every verification — no cache, REQ-NFR-VC-02).

TASK-09-04 — Public verification endpoint

  • Persona: security-engineer
  • File: server/vc/verification.py (new), server/__main__.py (extend — add route)
  • Content: GET /vc/verify/<credential_id> — public, unauthenticated (D-043). Fetch credential from SQLite, fetch issuer public key from verificationMethod URL, validate Ed25519 signature, check status list. Return {valid, status, issuer, credential, mastery, credentialTier: "formative", verifiedAt}. No PII beyond what the credential asserts.

TASK-09-05 — VC unit tests

  • Persona: security-engineer
  • File: tests/test_vc_issuer.py (new)
  • Content: key generation, sign/verify round-trip, tamper detection (flip a byte → verify fails), JCS canonicalization determinism, status list set/get, revocation invalidates verification.

TASK-09-06 — VC integration test (issue → verify round-trip + key rotation)

  • Persona: security-engineer
  • File: tests/test_vc_integration.py (new)
  • Content: issue a credential, GET /vc/verify/ → valid: true, credentialTier: formative. Revoke → GET → valid: false, status: revoked. Tamper payload → verify fails. Key rotation: old VC still verifies against archived public key.

TASK-09-07 — VC interop test (grill Axis 3 MUST #1 — external W3C verifier)

  • Persona: security-engineer
  • File: tests/test_vc_interop.py (new — staging-gated, requires external verifier dependency)
  • Content: verify a Praxis-issued VC against at least one external W3C VC verifier (e.g., digitalbazaar/vc-verifier or a JS @digitalcredentials/vc verifier via subprocess). Round-trip self-verification is insufficient for cryptographic claims. This is the grill's binding MUST — custom crypto code without interop verification is an unmitigated liability.

TASK-09-08 — Key-rotation operational drill (grill Axis 3 MUST #2)

  • Persona: security-engineer
  • File: tests/test_vc_key_rotation_drill.py (new)
  • Content: end-to-end operational drill — issue N VCs with key A, rotate to key B (archive A as superseded), issue M VCs with key B, verify all N+M VCs still verify (N against archived key A, M against active key B), revoke one of each, verify revocation. This is the one crypto procedure that, if broken, silently invalidates every credential ever issued.

Final Phase (P2) — Review + Audit + Milestone Ship

Branch: phase/02-final-review-ship → merged to milestone/v0.3-mastery-scoring → merged to main Ship: v0.1.5 (final patch = v0.3 milestone release) REQ-IDs covered: all v0.3 REQ-IDs (milestone-complete verification)

Tasks (delegated to ciagent-review + ciagent-audit + ciagent-ship)

  1. Run branch gate → create phase/02-final-review-ship
  2. ciagent-review — multi-persona review across P1; auto-apply P0 fixes, flag P1+
  3. ciagent-audit — reconstruction test, file discipline, branch hygiene, commit discipline
  4. ciagent-ship — merge phase/02 → milestone/v0.3 → main; tag v0.1.5; create release with full milestone summary
  5. Update REQUIREMENTS.md (all v0.3 REQ → complete), ROADMAP.md (v0.3 → complete; v0.4 = operator tier)
  6. Commit: docs(milestone): complete v0.3-mastery-scoring
  7. Clear checkpoint

REQ-ID Coverage Matrix (post-grill)

REQ-ID Phase Slice(s) Coverage
REQ-MAST-01 P1 SLICE-01, 03 rubric schema + scoring
REQ-MAST-02 P1 SLICE-07 mastery score + gate logic
REQ-MAST-03 P1 SLICE-09 VC issuer (formative-tier, SQLite-backed)
REQ-MAST-04 principle (accepted)
REQ-SCEN-02 P1 SLICE-04 IRT dynamic difficulty
REQ-SCEN-03 P1 SLICE-02, 06 scenario library
REQ-SCEN-04 P1 SLICE-02, 06 expert-authored format + AI variation hooks
REQ-PATH-02 P1 SLICE-05 6-week path structure
REQ-NFR-MAST-01 P1 SLICE-03 deterministic scoring
REQ-NFR-MAST-02 P1 SLICE-07, 09 gate auditability (SQLite)
REQ-NFR-VC-01 P1 SLICE-09 tamper-evidence + interop test (TASK-09-07)
REQ-NFR-VC-02 P1 SLICE-09 revocation latency (next verify call)
REQ-NFR-IRT-01 P1 SLICE-04 IRT <100ms

Deferred to v0.4 (operator tier — per grill Axis 2): REQ-DASH-01, REQ-AUTH-01, REQ-MT-01, REQ-MT-02, REQ-NFR-DASH-01, REQ-NFR-DASH-02, REQ-NFR-AUTH-01, REQ-NFR-MT-01.

v0.3 total: 13 REQ-IDs covered (7 functional + 6 NFR). 0 partial. 0 deferred within v0.3. 8 REQ-IDs deferred to v0.4.


Open Questions Deferred to EXECUTE

  1. R-VC-02 (validUntil): 3-year default, configurable per path. Confirm in SLICE-09.
  2. R-IRT-01 (cold start): Fall back to scenario.difficulty until ≥5 observations. Confirm in SLICE-04.
  3. R-MAST-03 (per-archetype weights): Ship refund/complaint weights only in v0.3 (static — dynamic branch-dependent re-weighting is a future feature per grill Axis 9 FIX). Confirm in SLICE-06.
  4. VC interop test dependency: TASK-09-07 requires an external W3C verifier. Confirm which verifier is available (digitalbazaar/vc-verifier or @digitalcredentials/vc) and whether it runs in CI or staging-only.