ci/README.md

# CIAgent — Continuous Intelligence

Fully autonomous, git-native AI-driven software engineering harness.

## Overview

CIAgent (Continuous Intelligence) is an autonomous-first software engineering harness that eliminates human-in-the-loop overhead while preserving the rigor of guided development. It receives a specification, resolves ambiguities through a single Clarify phase, then executes the full pipeline — research, plan, execute, verify — autonomously.

**The git log IS the project memory.** Every decision, escalation, lesson learned, and verification result is encoded in commit messages using structured `---ci---` YAML blocks. An agent's first impulse to gather context is `git log`, not file reads. Another agent with access to only commit messages (no code, no diffs) can reconstruct the project state completely.

## Intelligence Backends

CIAgent supports 5 intelligence backends. Set the appropriate environment variable and use `--backend` to select:

| Backend | Setup | Usage |
|---------|-------|-------|
| **OpenAI** | `export OPENAI_API_KEY=sk-...` | `ciagent run --backend openai` |
| **Anthropic** | `export ANTHROPIC_API_KEY=sk-ant-...` | `ciagent run --backend anthropic` |
| **Ollama Local** | `ollama serve` (localhost:11434) | `ciagent run --backend ollama-local` |
| **Ollama Cloud** | `export OLLAMA_CLOUD_API_KEY=...` | `ciagent run --backend ollama-cloud` |
| **Opencode** | `npm i -g opencode` | `ciagent run --backend opencode` |

Auto-detection (`--backend auto`, the default) tries: opencode → openai → ollama-local → ollama-cloud → anthropic.

## Installation

From source (package not yet published to npm):

```bash
git clone https://git.cloudinit.dev/continuous-intelligence/ci.git
cd ciagent
npm install
npm run build
npm link
```

## Quick Start

```bash
# Initialize from inline specification
ciagent init "Build a REST API for task management"

# Initialize from a specification file
ciagent init --spec ./specs/my-project.md

# Run the full autonomous pipeline
ciagent run --all

# Run a specific phase
ciagent run research
ciagent run plan
ciagent run execute
ciagent run verify

# Run with specific backends
ciagent run --all --backend openai
ciagent run --all --backend anthropic
ciagent run --all --backend ollama-local

# Execute an ad-hoc task
ciagent quick "Add authentication middleware"

# Check project status (reads from git log + branches)
ciagent status

# Discover improvement opportunities
ciagent ideate                        # Mechanical tier (always available)
ciagent ideate --category security    # Focus on specific categories
ciagent ideate --affected             # Cascade impact analysis
ciagent ideate --spec                 # Specification completeness analysis
ciagent ideate --external             # npm audit + dependency staleness
ciagent ideate --cross-project        # Cross-project pattern mining
ciagent ideate --project all          # Run across all active projects
ciagent ideate --output json          # JSON output mode
ciagent ideate --output markdown      # Markdown output mode

# Manage multiple projects
ciagent projects list                 # List all registered projects
ciagent projects add <slug> <name>    # Add a new project
ciagent projects set <slug>           # Set the active project

# Run with ideation stage
ciagent run --ideate                  # Insert IDEATE stage between RESEARCH and PLAN

# Run across all active projects
ciagent run --project all             # Execute pipeline for each project

# Review autonomous decisions (extracted from git log ---ci--- blocks)
ciagent audit
ciagent audit --verbose

# Debug an issue
ciagent debug "Tests failing on CI"

# Rollback a phase
ciagent rollback 1

# Ship a phase (verify, security, commit, tag)
ciagent ship 1
```

## Git-Native Architecture (v0.10.0)

### The Commit Schema

Every CIAgent-generated commit contains a `---ci---` YAML block with structured metadata:

```
feat(P01-01-02): create user registration endpoint

---ci---
phase: 1
milestone: v1.0
plan: 01-01
task: 01-01-02
status: execute
decisions:
  - id: D-003
    decision: Use bcrypt with 12 rounds for password hashing
    rationale: Industry standard; argon2 not available in target env
    confidence: 0.88
    alternatives: [argon2, scrypt]
requirements:
  covered: [AUTH-01]
---/ci---

- POST /auth/register validates email and password
- Checks for duplicate users
- Returns JWT token on success
```

### What Lives Where

| Where | What | Why |
|-------|------|-----|
| `.ciagent/config.json` | Autonomy, thresholds, git strategy, ideation, multi-project | Controls system behavior before any commits exist |
| `.ciagent/PROJECT.md` | Vision, core value, requirements, constraints, key decisions table | Long-lived strategic reference |
| `.ciagent/ARCHITECTURE.md` | System architecture, component boundaries, data flow | Long-lived technical reference |
| `.ciagent/ROADMAP.md` | Phase breakdown, milestone mapping, success criteria | Long-lived planning reference |
| `.ciagent/REQUIREMENTS.md` | v1/v2 requirements with REQ-IDs and traceability | Long-lived requirements reference |
| **Git commit bodies** | Decisions, escalations, lessons, compounds, verification results | Dynamic event stream — the audit trail |
| **Git branches** | Phase/milestone status | `phase/NN-slug` and `milestone/vX.X-slug` encode project structure |

### Branch Strategy

```
main
 └── milestone/v1.0-mvp
      ├── phase/01-authentication       # in progress if not merged
      ├── phase/02-task-management
      └── phase/03-realtime-notifications
```

- Branch exists + not merged = phase **in progress**
- Branch merged to milestone = phase **complete**
- Milestone branch merged to main = milestone **complete**

### Context Reconstruction Protocol

An agent starting a session gathers context in this order:

1. `git log --oneline -20` — recent activity
2. `git branch -a` — phase/milestone structure
3. `git log -1 --format="%b"` — latest `---ci---` block
4. `.ciagent/config.json` — autonomy + thresholds
5. `.ciagent/PROJECT.md` — vision + constraints (when needed)
6. `.ciagent/ROADMAP.md` — phase plan + success criteria (when needed)
7. `.ciagent/REQUIREMENTS.md` — REQ-IDs + traceability (when planning)
8. `.ciagent/ARCHITECTURE.md` — system structure (when researching)

Steps 1-3 take <1 second and provide 80% of the context needed.

### The Reconstruction Test

An agent with access to **only commit messages** (no code, no diffs, no `.ciagent/` files) can reconstruct:

| Reconstructable | How |
|---------------|-----|
| Project specification | Init commit body |
| Current phase | `---ci---.phase` field + branch status |
| Current milestone | Branch names + `---ci---.milestone` field |
| All decisions with rationale | `git log --grep="decisions:" --format="%b"` |
| Decision confidence | Each decision has `confidence: 0.XX` |
| Alternatives considered | Each decision has `alternatives: [...]` |
| Requirements coverage | `git log --grep="requirements:" --format="%b"` |
| Lessons learned | `git log --grep="lessons:" --format="%b"` |
| Compounded solutions | `git log --grep="compound:" --format="%b"` |
| Escalation history | `git log --grep="escalation:" --format="%b"` |

### Commit Types

In addition to conventional commit types, CIAgent uses:

| Type | When Used |
|------|-----------|
| `decision` | Autonomous decision logged (no code change) |
| `compound` | Compounded solution captured |
| `escalation` | Escalation raised or resolved |
| `verify` | Verification pass/fail |
| `wip` | Work-in-progress checkpoint |

## Autonomy Levels

| Level | Behavior |
|-------|----------|
| `full` | No human interaction after Clarify. Escalate only irreversible decisions. |
| `supervised` | Escalate on every Escalation Gate plus verification failures. |
| `guided` | Escalate on every Decision Gate. |

## Configuration

CIAgent uses `.ciagent/config.json` for project configuration:

```json
{
  "autonomy": {
    "level": "full",
    "escalation_hooks": ["deploy", "delete_data", "merge_to_main"],
    "clarify_budget": 10,
    "decision_confidence_threshold": 0.6,
    "max_revision_iterations": 3,
    "max_verification_retries": 2,
    "escalation_timeout_ms": 300000
  },
  "model_profile": "quality",
  "parallelization": {
    "enabled": true,
    "max_concurrent_agents": 5,
    "min_plans_for_parallel": 2,
    "max_concurrent_projects": 3
  },
  "verification": {
    "automated_only": true,
    "escalate_visual": true,
    "escalate_external_integration": true,
    "test_first": false
  },
  "security": {
    "auto_accept_low_severity": true,
    "auto_mitigate_medium_severity": true,
    "escalate_high_severity": true
  },
  "git": {
    "branching_strategy": "phase",
    "auto_commit": true,
    "auto_push": false
  },
  "ideation": {
    "enabled": true,
    "categories": ["security", "quality", "architecture", "coverage", "improvement"],
    "confidence_threshold": 0.6,
    "max_ideas": 20,
    "external_signals": {
      "npm_audit": true,
      "osv_advisories": true,
      "dependency_staleness": true
    },
    "cross_project": {
      "enabled": false,
      "similarity_weight": 0.5
    },
    "chaos": {
      "enabled": true,
      "scenarios": ["backend_unavailable", "requirement_change", "test_coverage_drop"]
    }
  }
}
```

## Architecture

### Pipeline

```
SPECIFY → CLARIFY → RESEARCH → IDEATE → PLAN → EXECUTE → TEST → VERIFY → COMPLETE
                 ↕               ↕        ↕         ↕     ↕      ↕
            (questions)    (auto-decide) (ideas) (auto-run) (auto-test) (auto-verify)
```

### Git-Native Core Modules

| Module | Purpose |
|--------|---------|
| `commit-parser` | `---ci---` YAML block extraction and parsing from commit messages |
| `commit-builder` | Structured commit message generation for all commit types |
| `git-context` | Project state reconstruction from `git log` + `git branch` |
| `git-branch` | Phase/milestone branch lifecycle management |
| `ciagent-files` | `.ciagent/` long-lived reference file management with update discipline |

### Decision Engine

Every autonomous decision is classified by confidence:
- **High (>0.85)**: Auto-decide, commit as `---ci---` block
- **Medium (0.60-0.85)**: Auto-decide with assumption logging, flag for review
- **Low (<0.60)**: Escalate to human

Decisions are committed to git as `decision` type commits. The audit trail is `git log --grep="decisions:"`.

### 19 Agents

| Agent | Role | CIAgent Modification |
|-------|------|----------------|
| orchestrator | Pipeline controller | Git-first context loading, `---ci---` commit generation |
| planner | Plan creation | Never sets `autonomous: false` |
| executor | Task execution | Never pauses for checkpoints |
| verifier | Output verification | Generates automated tests, not human UAT |
| researcher | Domain research | Logs assumptions, never flags for human |
| tester | Integration/e2e tests | Detects and runs existing test files, never writes tests |
| challenger | Plan stress-testing | Binding verdicts, only escalates <0.60 |
| security-auditor | Security audit | Auto-dispositions threats (STRIDE + CWE) |
| debugger | Bug fixing | Auto-fixes when confidence > threshold |
| code-reviewer | Code review | 3-persona review (security, performance, maintainability) |
| doc-writer | Documentation | Auto-updates ROADMAP/REQUIREMENTS/PROJECT.md |
| doc-verifier | Doc audit | Cross-checks docs vs. codebase (agent count, version, test count) |
| ideation-agent | Improvement ideas | Feeds uncovered requirements and repeated lessons into planning |
| roadmapper | Roadmap creation | Groups requirements by phase, generates success criteria |
| plan-checker | Plan validation | Checks structure, IDs, must-haves, wave order, requirement coverage |
| project-researcher | Ecosystem research | Detects frameworks, APIs, patterns, tooling from package.json |
| research-synthesizer | Research merge | Cross-references findings across .ciagent/ documents |
| solution-writer | Solution docs | Produces structured solution documents from plan + requirements |
| phase-researcher | Phase research | Extracts decisions, lessons, risks from git log for a specific phase |

### Ideation

CIAgent includes a built-in ideation engine that discovers improvement opportunities from git-native signals:

1. **Tier 1 — Mechanical**: Mines git history for uncovered requirements, repeated lessons, low-confidence decisions, escalation patterns, coverage gaps, architecture drift, and verification inversions
2. **Tier 2 — Backend-enriched**: When a backend is available, prioritizes mechanical findings and suggests novel improvements
3. **Tier 3 — Cross-project**: Mines patterns from other projects in the multi-project registry

```
ciagent ideate                          # All mechanical tiers
ciagent ideate --category security      # Security-focused ideas
ciagent ideate --affected               # Cascade impact from current changes
ciagent ideate --spec                   # Specification completeness analysis
ciagent ideate --external               # npm audit + OSV advisories
ciagent ideate --cross-project           # Cross-project pattern mining
ciagent ideate --project all             # Across all active projects
ciagent ideate --output json             # Machine-readable output
```

### Multi-Project

CIAgent supports multi-project workflows with `--project` flags:

```bash
# Initialize multiple projects
ciagent projects add task-api "Task API"
ciagent projects add auth-svc "Auth Service"

# Run ideation across all projects
ciagent ideate --project all

# Run pipeline for a specific project
ciagent run --project task-api

# Run pipeline across all projects
ciagent run --project all
```

Commit messages include project tracking in `---ci---` blocks:

```
---ci---
phase: 5
milestone: v0.10
project: task-api
status: execute
---/ci---
```

### Verification Layers

1. **Structural**: File existence, import/export wiring, no stubs
2. **Behavioral**: Test execution and requirement traceability — runs test framework, parses results, reports pass/fail per suite
3. **Security**: STRIDE threat pattern scanning with CWE mapping and confidence-based auto-disposition
4. **Code Quality**: 3-persona code review (security, performance, maintainability) with P0/P1/P2 findings

## Specification Format

```markdown
# Project: My Project

## Objective
Build a REST API for task management.

## Requirements
- User authentication (JWT-based)
- CRUD operations for tasks
- Real-time notifications

## Constraints
- Must use Node.js
- Must be production-ready

## Out of Scope
- Admin dashboard
- Mobile apps
```

## Escalation Protocol

When CIAgent cannot proceed autonomously:

1. **Irreversible Action**: Deploy, delete, merge to protected branch
2. **Verification Failure**: Tests pass but functional verification fails
3. **Low Confidence Decision**: Critical decision below threshold
4. **Security Escalation**: High-severity threat detected
5. **Specification Ambiguity**: Multiple valid interpretations

Each escalation is committed as an `escalation` type commit. Resolved escalations produce a follow-up commit with the resolution. The full escalation history is available via `git log --grep="escalation:"`.

## Current Limitations

- **Agent implementations**: All 18 non-orchestrator agents have intrinsic mechanical logic. Full LLM-powered agent behavior requires an intelligence backend (OpenAI, Anthropic, Ollama, or Opencode).
- **Package not published to npm**: Install from source only until a publishing pipeline is configured.

## Differences from Learnship

| Dimension | Learnship | CIAgent |
|-----------|-----------|-----|
| Project memory | `.planning/` directory files (legacy) | Git log + `---ci---` commit blocks |
| Audit trail | `.ciagent/audit/*.json` files (legacy) | `git log --grep="decisions:"` |
| State management | `STATE.md` + `STATE.md.json` (legacy) | Reconstructed from git on demand |
| Phase discovery | Read `.planning/phases/` directory (legacy) | `git branch -a \| grep phase/` |
| Human Interactions | 19+/lifecycle | 1-2/lifecycle |
| Decision Making | Human decides, agent implements | Agent decides, human reviews post-hoc |
| Verification | Human UAT | Automated tests + escalation |
| Specification | Multi-round conversation | Single spec file |

## Repository

[git.cloudinit.dev/continuous-intelligence/ci](https://git.cloudinit.dev/continuous-intelligence/ci)

## License

MIT