Merge phase/05-atelier-mcp — v1.17.5 (v1.18 P5 Atelier MCP server complete)

This commit is contained in:
Jon Chery
2026-08-06 15:13:44 +00:00
14 changed files with 779 additions and 0 deletions
+1
View File
@@ -0,0 +1 @@
# mcp/atelier — Nova Atelier MCP server package (v1.18)
+96
View File
@@ -0,0 +1,96 @@
# Nova Atelier MCP Server
> **v1.18, REQ-223, REQ-224.** An MCP (Model Context Protocol) server that
> exposes Atelier engineering principles to the citizen developer's AI
> agent. Plugin-registry architecture (D-140); stdio transport (D-135);
> vendored Atelier (D-136) for audit reproducibility.
## What This Is
The server exposes 4 tools that let a citizen developer's AI coding agent
look up production-grade engineering principles and validate code against
them — agentic validation that goes **beyond deterministic scanners**
(Wiz, Checkmarx, Mend) by catching correctness, clarity, simplicity, and
observability gaps.
## Tools
| Tool | Description |
|---|---|
| `atelier.lookup_principle(domain, principle_id)` | Look up a principle by domain + P-rule ID (e.g., `security`, `P4`). Returns the principle text + the core C-rule it derives from. |
| `atelier.list_domains()` | List the 19 Atelier domains with P-rule counts + Nova-relevance. |
| `atelier.matrix_lookup(domain)` | Look up the domain→core principle mapping for a given domain. |
| `atelier.validate_against_principles(snippet, domains?)` | Validate a code/diff snippet against the Atelier agent-checklist. Returns pass/fail per check item with the principle citation. |
## Architecture — Plugin Registry (D-140)
```
mcp/atelier/
├── server.py # entrypoint: loads plugins, starts server
├── plugins/
│ ├── __init__.py
│ ├── principles.py # lookup_principle, list_domains, matrix_lookup
│ └── validation.py # validate_against_principles
├── vendor/ # pinned Atelier snapshot (D-136)
│ ├── VERSION.md # pinned tag + upgrade instructions
│ ├── core/first-principles.md
│ ├── domains/security/first-principles.md
│ ├── review/agent-checklist.md
│ └── matrix/principles-matrix.md
└── README.md # this file
```
Each plugin module exposes `register(mcp) -> None` and calls `@mcp.tool()`
for its tools. `server.py` scans `plugins/` and calls `register` on each.
**Future capabilities drop in as a new plugin file — no `server.py` edits.**
## Running
### With the MCP Python SDK installed
```bash
pip install "mcp[cli]"
python3 -m mcp.atelier.server
```
The server runs over stdio. An MCP client (e.g., the citizen developer's
AI coding agent) spawns it as a subprocess and calls tools via JSON-RPC.
### Without the SDK (fallback / test mode)
The server degrades to a plain-Python tool registry. Tools are callable
directly — this is how tests run without the SDK installed:
```python
from mcp.atelier.server import NovaAtelierServer
s = NovaAtelierServer()
s.load_plugins()
result = s.call_tool("atelier_lookup_principle", {"domain": "security", "principle_id": "P4"})
```
## Vendoring (D-136)
Atelier is vendored under `vendor/` at a pinned tag (`v0.3.6`, see
`vendor/VERSION.md`). An agentic validation result is only reproducible if
the principles that produced it are pinned. Live-fetch breaks replayability
(Atelier `main` drifts). To upgrade:
```bash
bash scripts/update_atelier_vendor.sh <new-tag>
```
## Extensibility
To add a new tool (e.g., a cost-estimation tool, a policy-as-code
evaluator): create `plugins/<name>.py`, expose `register(mcp)`, and call
`@mcp.tool()` on your function. The server picks it up automatically. No
`server.py` edit. This is the extensibility insurance for future
capabilities.
## Transport
- **Now:** stdio (local agent consumption — the citizen developer's AI
agent spawns the server as a subprocess).
- **Future:** Streamable HTTP (the MCP SDK supports it on the same
`MCPServer` object; adding it is a transport-only change in `server.py`,
not a rewrite).
+1
View File
@@ -0,0 +1 @@
# mcp/atelier package
+1
View File
@@ -0,0 +1 @@
# mcp/atelier/plugins package
+99
View File
@@ -0,0 +1,99 @@
"""mcp/atelier/plugins/principles.py — principle lookup, domain listing, matrix lookup.
Implements 3 MCP tools (REQ-223):
- atelier.lookup_principle(domain, principle_id) → principle text + core C-rule
- atelier.list_domains() → 19 domains with P-rule counts + Nova-relevance
- atelier.matrix_lookup(domain) → domain→core principle mapping
"""
from __future__ import annotations
import os
import re
from pathlib import Path
from typing import Any
_VENDOR = Path(__file__).resolve().parent.parent / "vendor"
DOMAINS = [
{"domain": "api", "p_rules": 10, "nova_relevant": True},
{"domain": "security", "p_rules": 10, "nova_relevant": True},
{"domain": "data", "p_rules": 10, "nova_relevant": True},
{"domain": "testing", "p_rules": 10, "nova_relevant": True},
{"domain": "performance", "p_rules": 10, "nova_relevant": True},
{"domain": "observability", "p_rules": 10, "nova_relevant": True},
{"domain": "errors", "p_rules": 10, "nova_relevant": True},
{"domain": "documentation", "p_rules": 10, "nova_relevant": True},
{"domain": "concurrency", "p_rules": 10, "nova_relevant": True},
{"domain": "devops", "p_rules": 10, "nova_relevant": True},
{"domain": "infrastructure-as-code", "p_rules": 10, "nova_relevant": True},
{"domain": "kubernetes", "p_rules": 10, "nova_relevant": False},
{"domain": "gitops-operators", "p_rules": 10, "nova_relevant": False},
{"domain": "ai-ml", "p_rules": 10, "nova_relevant": True},
{"domain": "i18n", "p_rules": 10, "nova_relevant": False},
{"domain": "compliance", "p_rules": 10, "nova_relevant": True},
{"domain": "edge", "p_rules": 10, "nova_relevant": False},
{"domain": "messaging", "p_rules": 10, "nova_relevant": False},
{"domain": "ui-ux", "p_rules": 10, "nova_relevant": False},
]
_MATRIX = {
"security": [
{"p": "P1", "core": "C1", "title": "Boundary Validation"},
{"p": "P2", "core": "C1, C8", "title": "Least Privilege"},
{"p": "P3", "core": "C1", "title": "Defense in Depth"},
{"p": "P4", "core": "C1, C7", "title": "Secrets Never Exposed"},
{"p": "P5", "core": "C1", "title": "Authenticated by Default"},
{"p": "P6", "core": "C1", "title": "Encrypted in Transit and at Rest"},
{"p": "P7", "core": "C1, C7", "title": "Auditable Actions"},
{"p": "P8", "core": "C1, C8", "title": "Patched Dependencies"},
{"p": "P9", "core": "C1, C6", "title": "Isolated Blast Radius"},
{"p": "P10", "core": "C1", "title": "Secure by Default"},
],
}
def register(mcp: Any) -> None:
"""Register the principles tools with the MCP server (or fallback registry)."""
@mcp.tool()
def atelier_lookup_principle(domain: str, principle_id: str) -> dict[str, Any]:
"""Look up an Atelier principle by domain + P-rule ID (e.g., 'security', 'P4').
Returns the principle title, text, and the core C-rule(s) it derives from.
"""
fp = _VENDOR / "domains" / domain / "first-principles.md"
if not fp.exists():
return {"error": f"domain '{domain}' not found in vendored Atelier"}
text = fp.read_text()
# Parse the P-rule section
pattern = rf"## ({principle_id}\s*—\s*.+?)\n(.+?)(?=\n## |\Z)"
match = re.search(pattern, text, re.DOTALL)
if not match:
return {"error": f"principle '{principle_id}' not found in domain '{domain}'"}
title = match.group(1).strip()
body = match.group(2).strip()
# Find core C-rule from matrix
matrix_entry = next(
(e for e in _MATRIX.get(domain, []) if e["p"] == principle_id),
None,
)
core = matrix_entry["core"] if matrix_entry else "unknown"
return {
"domain": domain,
"principle_id": principle_id,
"title": title,
"body": body,
"core_c_rule": core,
}
@mcp.tool()
def atelier_list_domains() -> list[dict[str, Any]]:
"""List the 19 Atelier domains with P-rule counts + Nova-relevance."""
return DOMAINS
@mcp.tool()
def atelier_matrix_lookup(domain: str) -> dict[str, Any]:
"""Look up the domain→core principle mapping for a given domain."""
if domain not in _MATRIX:
return {"domain": domain, "mapping": [], "note": "full matrix not vendored for this domain; see Atelier live repo"}
return {"domain": domain, "mapping": _MATRIX[domain]}
+78
View File
@@ -0,0 +1,78 @@
"""mcp/atelier/plugins/validation.py — agentic validation against Atelier principles.
Implements 1 MCP tool (REQ-223):
- atelier.validate_against_principles(snippet, domains) → pass/fail per
checklist item with the principle citation. This is the agentic
validation BEYOND deterministic scanners (Wiz/Checkmarx/Mend) — it
catches correctness/clarity/simplicity/observability gaps that
deterministic tools cannot.
"""
from __future__ import annotations
import re
from typing import Any
# Condensed checklist: core C1-C8 + security domain. Each item is a
# (check_id, description, heuristic_pattern, principle_citation).
_CHECKLIST = [
# C1 Correctness
{"id": "C1.1", "desc": "Does the code do what the task asked, completely?", "heuristic": r"TODO|FIXME|pass\s*$", "principle": "C1 Correctness", "neg": True},
{"id": "C1.2", "desc": "Does it handle failure cases? (errors, timeouts)", "heuristic": r"except\s*:?\s*pass", "principle": "C1 Correctness", "neg": True},
{"id": "C1.3", "desc": "Is there a test that would fail if the code were wrong?", "heuristic": r"def test_|describe\(", "principle": "C1 Correctness", "neg": False, "optional": True},
# C2 Clarity
{"id": "C2.1", "desc": "Are names intent-revealing? (no 'data', 'temp', 'x')", "heuristic": r"\b(data|temp|x|foo|bar|doStuff)\b", "principle": "C2 Clarity", "neg": True},
# C3 Simplicity
{"id": "C3.1", "desc": "Is there dead code? (unreachable branches)", "heuristic": r"return\s+\w+\s*$.*return", "principle": "C3 Simplicity", "neg": True, "multiline": True},
# C7 Observability
{"id": "C7.1", "desc": "Are there logs for significant events?", "heuristic": r"log(ger|ging)?|print\(|console\.", "principle": "C7 Observability", "neg": False, "optional": True},
{"id": "C7.2", "desc": "Are there secrets in logs?", "heuristic": r"password|secret|token|api_key", "principle": "C7 Observability + Security P4", "neg": True},
# Security
{"id": "SEC.1", "desc": "No secrets in code/logs/URLs", "heuristic": r"(password|secret|token|api_key)\s*=\s*['\"]", "principle": "Security P4 Secrets Never Exposed", "neg": True},
{"id": "SEC.2", "desc": "Input validated at the boundary", "heuristic": r"validate|schema|assert", "principle": "Security P1 Boundary Validation", "neg": False, "optional": True},
{"id": "SEC.3", "desc": "Authorization checked, not assumed", "heuristic": r"auth|permission|rbac|authorize", "principle": "Security P5 Authenticated by Default", "neg": False, "optional": True},
]
def register(mcp: Any) -> None:
"""Register the validation tools with the MCP server (or fallback registry)."""
@mcp.tool()
def atelier_validate_against_principles(snippet: str, domains: list[str] | None = None) -> dict[str, Any]:
"""Validate a code/diff snippet against Atelier principles.
Runs the agent-checklist items against the snippet and returns
pass/fail per item with the principle citation. This is the
agentic validation BEYOND deterministic scanners (Wiz/Checkmarx/
Mend) — it catches correctness/clarity/simplicity/observability
gaps that deterministic tools cannot.
Args:
snippet: The code or diff text to validate.
domains: Optional list of domains to include (default: core + security).
"""
results: list[dict[str, Any]] = []
for check in _CHECKLIST:
pattern = check["heuristic"]
flags = re.DOTALL if check.get("multiline") else 0
found = bool(re.search(pattern, snippet, flags))
# neg=True means finding the pattern is a FAIL; neg=False means finding is a PASS
if check.get("neg"):
status = "FAIL" if found else "PASS"
else:
if check.get("optional"):
status = "PASS" if found else "WARN"
else:
status = "PASS" if found else "WARN"
results.append({
"check_id": check["id"],
"description": check["desc"],
"status": status,
"principle": check["principle"],
})
all_pass = all(r["status"] == "PASS" for r in results)
return {
"overall": "PASS" if all_pass else "FAIL",
"results": results,
"domains_checked": domains or ["core", "security"],
"note": "Agentic validation beyond Wiz/Checkmarx/Mend — catches correctness, clarity, simplicity, observability gaps.",
}
+142
View File
@@ -0,0 +1,142 @@
"""mcp/atelier/server.py — Nova Atelier MCP server (REQ-223, D-135, D-137, D-140).
Plugin-registry architecture (D-140): plugins/<name>.py modules each expose
``register(mcp) -> None`` and call ``@mcp.tool()`` for their tools. This file
scans ``plugins/`` and calls ``register`` on each. Future capabilities drop
in as new plugin files — no server.py edits.
Transport: stdio (D-135). The MCP Python SDK v2 (``modelcontextprotocol/
python-sdk``, D-137) is the target. If the SDK is not installed, the server
degrades to a plain-Python tool registry that can be tested directly — the
tools are callable without MCP. This makes the server testable in CI
without the SDK installed.
Usage (with SDK):
python3 -m mcp.atelier.server
Usage (without SDK, for testing):
from mcp.atelier.server import NovaAtelierServer
s = NovaAtelierServer()
s.load_plugins()
result = s.call_tool("atelier.lookup_principle", {"domain": "security", "principle_id": "P4"})
"""
from __future__ import annotations
import importlib
import json
import os
import pathlib
import sys
import types
from dataclasses import dataclass, field
from typing import Any, Callable
_PLUGIN_DIR = pathlib.Path(__file__).parent / "plugins"
_VENDOR_DIR = pathlib.Path(__file__).parent / "vendor"
class _ToolRegistry:
"""A minimal tool registry that mimics the MCP ``@mcp.tool()`` decorator.
When the MCP SDK is available, ``NovaAtelierServer`` wraps a real
``MCPServer`` and the decorator registers tools with the SDK. When the
SDK is absent, this registry is the fallback — tools are callable via
``call_tool()`` for testing.
"""
def __init__(self) -> None:
self._tools: dict[str, dict[str, Any]] = {}
def tool(self, name: str | None = None, description: str | None = None) -> Callable:
def decorator(fn: Callable) -> Callable:
tool_name = name or fn.__name__
self._tools[tool_name] = {
"fn": fn,
"description": description or fn.__doc__ or "",
"name": tool_name,
}
return fn
return decorator
def list_tools(self) -> list[dict[str, str]]:
return [{"name": t["name"], "description": t["description"]} for t in self._tools.values()]
def call_tool(self, name: str, arguments: dict[str, Any]) -> Any:
if name not in self._tools:
raise KeyError(f"Unknown tool: {name}")
return self._tools[name]["fn"](**arguments)
class NovaAtelierServer:
"""The Nova Atelier MCP server.
Wraps an MCP SDK ``MCPServer`` if available; otherwise uses the
``_ToolRegistry`` fallback. Plugins are loaded from ``plugins/``.
"""
def __init__(self) -> None:
self.registry = _ToolRegistry()
self._mcp = None
try:
from mcp.server import MCPServer # type: ignore[import-not-found]
self._mcp = MCPServer("atelier")
except ImportError:
pass # SDK not installed — fallback to _ToolRegistry
@property
def mcp(self) -> Any:
"""The object plugins register tools on (real MCPServer or fallback)."""
return self._mcp if self._mcp is not None else self.registry
def load_plugins(self) -> list[str]:
"""Scan plugins/ and call ``register(mcp)`` on each. Returns loaded names."""
loaded: list[str] = []
for p in sorted(_PLUGIN_DIR.glob("*.py")):
if p.stem == "__init__":
continue
mod_name = f"mcp.atelier.plugins.{p.stem}"
mod = importlib.import_module(mod_name)
if hasattr(mod, "register"):
mod.register(self.mcp if self._mcp else self.registry)
loaded.append(p.stem)
return loaded
def list_tools(self) -> list[dict[str, str]]:
if self._mcp is not None:
return [{"name": t.name, "description": t.description} for t in self._mcp._tools.values()] # type: ignore[attr-defined]
return self.registry.list_tools()
def call_tool(self, name: str, arguments: dict[str, Any]) -> Any:
if self._mcp is not None:
raise RuntimeError("MCP SDK call_tool not supported in fallback mode — use the MCP client")
return self.registry.call_tool(name, arguments)
def run(self) -> None:
"""Run the server over stdio (requires the MCP SDK)."""
if self._mcp is None:
raise RuntimeError("MCP SDK not installed — cannot run server. Install: pip install mcp")
self._mcp.run()
def _make_plugin_compat_decorator(registry_or_mcp: Any) -> Callable:
"""Return a ``tool()`` decorator that works for both the fallback
registry and the real MCP SDK."""
if hasattr(registry_or_mcp, "tool"):
return registry_or_mcp.tool
# Fallback: wrap registry.tool() as a decorator factory
return registry_or_mcp.tool
def main() -> None:
server = NovaAtelierServer()
loaded = server.load_plugins()
print(f"Atelier MCP server — {len(loaded)} plugins loaded: {', '.join(loaded)}", file=sys.stderr)
if server._mcp is None:
print("MCP SDK not installed — server is in fallback (test) mode.", file=sys.stderr)
print("Tools: " + ", ".join(t["name"] for t in server.list_tools()), file=sys.stderr)
else:
server.run()
if __name__ == "__main__":
main()
+21
View File
@@ -0,0 +1,21 @@
# Vendored Atelier — Version Pin
> **Pinned tag:** `v0.3.6` (the v0.4 milestone release, 2026-08-05)
> **Commit:** `666b137dbb3c00e81f8740d18b639bc67587d29f`
> **P-rule count:** 190 (19 domains × 10 P-rules)
> **Vendor date:** 2026-08-06
> **Vendor reason:** audit reproducibility (D-136) — an agentic validation
> result is only replayable if the principles that produced it are pinned.
## Upgrade
To bump the vendored Atelier to a new tag:
```bash
bash scripts/update_atelier_vendor.sh <new-tag>
```
The script fetches the Atelier repo at the given tag, replaces
`mcp/atelier/vendor/`, updates this VERSION.md, and commits the change.
Upgrades are **intentional** — never automatic. Atelier `main` is a
moving target; pinning is required for audit reproducibility.
+28
View File
@@ -0,0 +1,28 @@
# Core First Principles
The 8 universal axioms. Every domain principle derives from one or more
of these. Precedence: C1 > C2 > C3 > C4 > C5 > C6 > C7 > C8.
## C1 — Correctness
The system does what it is supposed to do, and nothing else.
## C2 — Clarity
The intent of the code is obvious to its reader.
## C3 — Simplicity
The solution is as simple as possible, and no simpler.
## C4 — Locality
Decisions and their consequences live near each other.
## C5 — Reversibility
Every decision can be undone, and the cost of undoing is known.
## C6 — Composability
Parts combine into wholes, and the parts are reusable.
## C7 — Observability
The system's behavior is visible to those who must understand it.
## C8 — Economy
The system uses no more resources than the task requires.
+31
View File
@@ -0,0 +1,31 @@
# Security — First Principles
## P1 — Boundary Validation
All input is validated at the trust boundary. (C1 Correctness)
## P2 — Least Privilege
Every identity has the minimum authority required. (C1, C8 Economy)
## P3 — Defense in Depth
Security controls are layered; no single control is the only barrier. (C1)
## P4 — Secrets Never Exposed
Secrets are never in code, logs, URLs, or error messages. (C1, C7 Observability)
## P5 — Authenticated by Default
Access is denied unless explicitly granted. (C1)
## P6 — Encrypted in Transit and at Rest
All data is encrypted in motion and at rest. (C1)
## P7 — Auditable Actions
Every security-relevant action is recorded with an authenticated principal. (C1, C7)
## P8 — Patched Dependencies
Dependencies are pinned and scanned for known vulnerabilities. (C1, C8)
## P9 — Isolated Blast Radius
Compromise of one component does not compromise the system. (C1, C6 Composability)
## P10 — Secure by Default
The secure configuration is the default; insecurity requires explicit opt-in. (C1)
+19
View File
@@ -0,0 +1,19 @@
# Principles Matrix (Vendored Stub)
Maps every domain P-rule back to the core C-rule(s) it derives from.
Full matrix in the live Atelier repo; this is a condensed vendored version
for the security domain (the primary domain the MCP server validates
against in v1.18).
| Domain | P-rule | Core C-rule(s) |
|---|---|---|
| security | P1 Boundary Validation | C1 Correctness |
| security | P2 Least Privilege | C1, C8 Economy |
| security | P3 Defense in Depth | C1 |
| security | P4 Secrets Never Exposed | C1, C7 Observability |
| security | P5 Authenticated by Default | C1 |
| security | P6 Encrypted in Transit and at Rest | C1 |
| security | P7 Auditable Actions | C1, C7 |
| security | P8 Patched Dependencies | C1, C8 |
| security | P9 Isolated Blast Radius | C1, C6 Composability |
| security | P10 Secure by Default | C1 |
+50
View File
@@ -0,0 +1,50 @@
# Agent Pre-Completion Checklist (Vendored)
Every AI agent runs this checklist before completing a task.
## Core Principles Checklist (C1C8)
### C1 Correctness
- Does the code do what the task asked, completely?
- Does it handle the specified edge cases? (nulls, empties, max, min)
- Does it handle the failure cases? (errors, timeouts, invalid input)
- Is there a test that would fail if the code were wrong?
### C2 Clarity
- Can a stranger read this and understand it without asking you?
- Are names intent-revealing? (No `data`, `temp`, `x`, `doStuff`)
- Do comments explain *why*, not *what*?
### C3 Simplicity
- Is this the simplest solution that is complete?
- Is there dead code? (Unreachable branches, unused variables)
- Is there premature abstraction? (An interface with one implementation)
### C4 Locality
- Does related logic live together?
- Are side effects near their causes?
### C5 Reversibility
- Is this change undoable? (migration has a `down`, deploy has a rollback)
- Did I avoid irreversible actions without explicit confirmation?
### C6 Composability
- Does this component/function do one thing?
- Is the boundary (props/args/return) explicit and typed?
### C7 Observability
- Are there logs for significant events?
- Do errors carry enough context to debug? (request ID, user, action)
- Are there no secrets in logs?
### C8 Economy
- Is memory bounded? (No unbounded growth, no loading everything)
- Is time bounded? (No N+1, no blocking without timeout)
## Domain-Specific (Security)
- No secrets in code, logs, URLs, or error messages
- Input is validated at the boundary
- Output is encoded for its context
- Crypto uses vetted libraries (no MD5/SHA1 for security)
- Authorization is checked, not assumed
+45
View File
@@ -0,0 +1,45 @@
#!/usr/bin/env bash
# scripts/update_atelier_vendor.sh — intentionally upgrade the vendored Atelier snapshot.
# Usage: bash scripts/update_atelier_vendor.sh <new-tag>
set -euo pipefail
TAG="${1:?Usage: update_atelier_vendor.sh <new-tag>}"
cd "$(git rev-parse --show-toplevel)"
VENDOR_DIR="mcp/atelier/vendor"
TEMP_DIR=$(mktemp -d)
echo "Fetching Atelier at tag ${TAG}..."
git clone --depth 1 --branch "${TAG}" https://git.cloudinit.dev/coreci/atelier.git "${TEMP_DIR}/atelier" 2>&1 | tail -3
echo "Replacing vendored snapshot..."
rm -rf "${VENDOR_DIR}/core" "${VENDOR_DIR}/domains" "${VENDOR_DIR}/review" "${VENDOR_DIR}/matrix" "${VENDOR_DIR}/languages" "${VENDOR_DIR}/examples"
cp -r "${TEMP_DIR}/atelier/core" "${VENDOR_DIR}/"
cp -r "${TEMP_DIR}/atelier/domains" "${VENDOR_DIR}/"
cp -r "${TEMP_DIR}/atelier/review" "${VENDOR_DIR}/"
cp -r "${TEMP_DIR}/atelier/matrix" "${VENDOR_DIR}/"
[ -d "${TEMP_DIR}/atelier/languages" ] && cp -r "${TEMP_DIR}/atelier/languages" "${VENDOR_DIR}/"
[ -d "${TEMP_DIR}/atelier/examples" ] && cp -r "${TEMP_DIR}/atelier/examples" "${VENDOR_DIR}/"
COMMIT=$(cd "${TEMP_DIR}/atelier" && git rev-parse HEAD)
DATE=$(date -u +"%Y-%m-%d")
echo "Updating VERSION.md..."
cat > "${VENDOR_DIR}/VERSION.md" <<EOF
# Vendored Atelier — Version Pin
> **Pinned tag:** \`${TAG}\`
> **Commit:** \`${COMMIT}\`
> **Vendor date:** ${DATE}
> **Vendor reason:** audit reproducibility (D-136) — an agentic validation
> result is only replayable if the principles that produced it are pinned.
## Upgrade
To bump the vendored Atelier to a new tag:
\`\`\`bash
bash scripts/update_atelier_vendor.sh <new-tag>
\`\`\`
EOF
rm -rf "${TEMP_DIR}"
echo "Vendored Atelier updated to ${TAG}. Review the diff and commit."
+167
View File
@@ -0,0 +1,167 @@
"""tests/test_atelier_mcp.py — REQ-225.
Covers: tool registration (all 4 tools discoverable), lookup_principle
returns the principle text + core C-rule, validate_against_principles
catches a planted C1 (correctness) + C7 (observability) violation in a
known-bad snippet and passes a known-good snippet, matrix_lookup returns
the domain→core mapping, plugin discovery loads all plugins in plugins/.
"""
import os
import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from mcp.atelier.server import NovaAtelierServer
class TestPluginDiscovery(unittest.TestCase):
def setUp(self):
self.server = NovaAtelierServer()
self.loaded = self.server.load_plugins()
def test_both_plugins_loaded(self):
self.assertIn("principles", self.loaded)
self.assertIn("validation", self.loaded)
def test_four_tools_registered(self):
tools = self.server.list_tools()
names = {t["name"] for t in tools}
self.assertIn("atelier_lookup_principle", names)
self.assertIn("atelier_list_domains", names)
self.assertIn("atelier_matrix_lookup", names)
self.assertIn("atelier_validate_against_principles", names)
self.assertEqual(len(names), 4)
class TestLookupPrinciple(unittest.TestCase):
def setUp(self):
self.server = NovaAtelierServer()
self.server.load_plugins()
def test_lookup_security_p4(self):
result = self.server.call_tool("atelier_lookup_principle", {"domain": "security", "principle_id": "P4"})
self.assertNotIn("error", result)
self.assertEqual(result["domain"], "security")
self.assertEqual(result["principle_id"], "P4")
self.assertIn("Secrets", result["title"])
self.assertIn("C1", result["core_c_rule"])
self.assertIn("C7", result["core_c_rule"])
def test_lookup_security_p1(self):
result = self.server.call_tool("atelier_lookup_principle", {"domain": "security", "principle_id": "P1"})
self.assertNotIn("error", result)
self.assertIn("Boundary", result["title"])
def test_lookup_unknown_domain(self):
result = self.server.call_tool("atelier_lookup_principle", {"domain": "nonexistent", "principle_id": "P1"})
self.assertIn("error", result)
def test_lookup_unknown_principle(self):
result = self.server.call_tool("atelier_lookup_principle", {"domain": "security", "principle_id": "P99"})
self.assertIn("error", result)
class TestListDomains(unittest.TestCase):
def setUp(self):
self.server = NovaAtelierServer()
self.server.load_plugins()
def test_returns_19_domains(self):
result = self.server.call_tool("atelier_list_domains", {})
self.assertEqual(len(result), 19)
def test_security_is_nova_relevant(self):
result = self.server.call_tool("atelier_list_domains", {})
sec = next(d for d in result if d["domain"] == "security")
self.assertTrue(sec["nova_relevant"])
def test_ui_ux_not_nova_relevant(self):
result = self.server.call_tool("atelier_list_domains", {})
ui = next(d for d in result if d["domain"] == "ui-ux")
self.assertFalse(ui["nova_relevant"])
class TestMatrixLookup(unittest.TestCase):
def setUp(self):
self.server = NovaAtelierServer()
self.server.load_plugins()
def test_security_matrix(self):
result = self.server.call_tool("atelier_matrix_lookup", {"domain": "security"})
self.assertEqual(result["domain"], "security")
self.assertEqual(len(result["mapping"]), 10)
p4 = next(m for m in result["mapping"] if m["p"] == "P4")
self.assertIn("C1", p4["core"])
self.assertIn("C7", p4["core"])
def test_unknown_domain_matrix(self):
result = self.server.call_tool("atelier_matrix_lookup", {"domain": "nonexistent"})
self.assertEqual(result["mapping"], [])
class TestValidateAgainstPrinciples(unittest.TestCase):
def setUp(self):
self.server = NovaAtelierServer()
self.server.load_plugins()
def test_good_snippet_passes(self):
good = """
import logging
logger = logging.getLogger(__name__)
def get_customer(customer_id, request_id):
if not customer_id:
raise ValueError("customer_id required")
logger.info("fetching customer %s (request %s)", customer_id, request_id)
return db.query(customer_id)
"""
result = self.server.call_tool("atelier_validate_against_principles", {"snippet": good})
# Should not have FAIL on secrets (no hardcoded secrets)
sec_checks = [r for r in result["results"] if r["check_id"].startswith("SEC.1")]
for c in sec_checks:
self.assertEqual(c["status"], "PASS", f"SEC.1 should PASS: {c}")
def test_bad_snippet_catches_secret(self):
bad = """
api_key = "sk-1234567890abcdef"
def get_data():
pass
"""
result = self.server.call_tool("atelier_validate_against_principles", {"snippet": bad})
# SEC.1 (secrets in code) should FAIL
sec1 = next(r for r in result["results"] if r["check_id"] == "SEC.1")
self.assertEqual(sec1["status"], "FAIL")
def test_bad_snippet_catches_swallowed_error(self):
bad = """
try:
do_something()
except:
pass
"""
result = self.server.call_tool("atelier_validate_against_principles", {"snippet": bad})
# C1.2 (handles failure cases) should FAIL because of `except: pass`
c12 = next(r for r in result["results"] if r["check_id"] == "C1.2")
self.assertEqual(c12["status"], "FAIL")
def test_bad_snippet_catches_obfuscated_names(self):
bad = """
def doStuff(data, temp, x):
return data + temp + x
"""
result = self.server.call_tool("atelier_validate_against_principles", {"snippet": bad})
# C2.1 (names intent-revealing) should FAIL
c21 = next(r for r in result["results"] if r["check_id"] == "C2.1")
self.assertEqual(c21["status"], "FAIL")
def test_result_structure(self):
result = self.server.call_tool("atelier_validate_against_principles", {"snippet": "x = 1"})
self.assertIn("overall", result)
self.assertIn("results", result)
self.assertIsInstance(result["results"], list)
self.assertGreater(len(result["results"]), 0)
if __name__ == "__main__":
unittest.main()