path: root/makima-architecture.md

# Makima Architecture Analysis

## Executive Summary

Makima is a distributed task orchestration system for managing AI coding agents (primarily Claude Code instances). It follows a client-server architecture with daemons running on local machines that execute tasks, while a central server coordinates work through contracts. The system already implements several patterns similar to ralph, including completion gates, autonomous loop mode, and circuit breakers.

---

## 1. Current Architecture Overview

### 1.1 High-Level Architecture

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                            MAKIMA SERVER (Rust)                              │
│  ┌─────────────┐  ┌──────────────┐  ┌─────────────┐  ┌──────────────────┐   │
│  │   REST API  │  │  WebSocket   │  │  PostgreSQL │  │   LLM Tools      │   │
│  │   Handlers  │  │    Hub       │  │     DB      │  │ (Chat, Analysis) │   │
│  └─────────────┘  └──────────────┘  └─────────────┘  └──────────────────┘   │
└─────────────────────────────────────────────────────────────────────────────┘
                                    │
                    ┌───────────────┼───────────────┐
                    ▼               ▼               ▼
            ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
            │   DAEMON 1   │ │   DAEMON 2   │ │   DAEMON N   │
            │  (Worker)    │ │  (Worker)    │ │  (Worker)    │
            ├──────────────┤ ├──────────────┤ ├──────────────┤
            │ Task Manager │ │ Task Manager │ │ Task Manager │
            │   Worktree   │ │   Worktree   │ │   Worktree   │
            │   Manager    │ │   Manager    │ │   Manager    │
            │   Process    │ │   Process    │ │   Process    │
            │   Manager    │ │   Manager    │ │   Manager    │
            └──────────────┘ └──────────────┘ └──────────────┘
                    │               │               │
                    ▼               ▼               ▼
            ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
            │ Claude Code  │ │ Claude Code  │ │ Claude Code  │
            │  Instances   │ │  Instances   │ │  Instances   │
            └──────────────┘ └──────────────┘ └──────────────┘
```

### 1.2 Core Components

#### Server-Side (`makima/src/server/`)

| Component | Location | Responsibility |
|-----------|----------|----------------|
| **REST API** | `handlers/*.rs` | HTTP endpoints for contracts, tasks, files, mesh operations |
| **WebSocket Hub** | `handlers/mesh_daemon.rs`, `mesh_ws.rs` | Real-time communication with daemons |
| **Database** | `../db/` | PostgreSQL via sqlx for persistent state |
| **Authentication** | `auth.rs` | API key and JWT authentication |
| **LLM Integration** | `../llm/` | Claude/Groq clients, tool execution |

#### Daemon-Side (`makima/src/daemon/`)

| Component | Location | Responsibility |
|-----------|----------|----------------|
| **Task Manager** | `task/manager.rs` | Task lifecycle, concurrency control |
| **Task State** | `task/state.rs` | State machine for task progression |
| **Completion Gate** | `task/completion_gate.rs` | Autonomous loop termination logic |
| **Process Manager** | `process/` | Spawns and manages Claude Code subprocesses |
| **Worktree Manager** | `worktree/` | Git worktree isolation for tasks |
| **WebSocket Client** | `ws/` | Bidirectional communication with server |
| **Local DB** | `db/local.rs` | SQLite for crash recovery |
| **TUI** | `tui/` | Interactive terminal interface |

---

## 2. Key Components and Responsibilities

### 2.1 Contract System

Contracts are the top-level organizational unit representing a body of work:

```rust
// From db/models.rs
pub struct Contract {
    pub id: Uuid,
    pub name: String,
    pub contract_type: String,    // "simple" or "specification"
    pub phase: String,            // research, specify, plan, execute, review
    pub status: String,           // active, completed, archived
    pub supervisor_task_id: Option<Uuid>,
    pub autonomous_loop: bool,    // Enable auto-restart on incomplete
    pub phase_guard: bool,        // Require user approval for phase transitions
}
```

**Contract Types:**
- `simple`: Plan → Execute workflow
- `specification`: Research → Specify → Plan → Execute → Review

### 2.2 Task Orchestration

Tasks represent individual units of work executed by Claude Code:

```rust
// Simplified from db/models.rs
pub struct Task {
    pub id: Uuid,
    pub contract_id: Option<Uuid>,
    pub parent_task_id: Option<Uuid>,
    pub is_supervisor: bool,
    pub status: String,  // pending, running, paused, blocked, done, failed
    pub plan: String,
    pub daemon_id: Option<Uuid>,
    pub continue_from_task_id: Option<Uuid>,  // For task continuation chains
    pub conversation_state: Option<serde_json::Value>,  // For resumption
}
```

**Task Hierarchy:**
1. **Supervisor Tasks**: Long-running orchestrators that spawn worker tasks
2. **Worker Tasks**: Execute specific implementation work
3. **Subtask Chains**: Tasks can continue from other tasks' worktrees

### 2.3 Task State Machine

```
                    ┌────────────────┐
                    │  Initializing  │
                    └───────┬────────┘
                            │
                    ┌───────▼────────┐
                    │    Starting    │
                    └───────┬────────┘
                            │
            ┌───────────────▼───────────────┐
            │           Running             │
            └───────┬───────┬───────┬───────┘
                    │       │       │
        ┌───────────▼─┐ ┌───▼───┐ ┌─▼────────┐
        │   Paused    │ │Blocked│ │Completed │
        └─────────────┘ └───────┘ └──────────┘
                    │       │
                    ▼       ▼
            ┌──────────────────────┐
            │  Failed/Interrupted  │
            └──────────────────────┘
```

### 2.4 Worktree Isolation

Each task gets its own git worktree, providing:
- Complete isolation from other tasks
- Ability to merge changes via git
- Checkpointing via git commits

```rust
// From daemon/worktree/mod.rs
pub struct WorktreeInfo {
    pub path: PathBuf,
    pub branch: String,
    pub task_id: Uuid,
}
```

---

## 3. Existing Context Management Mechanisms

### 3.1 Completion Gate (Ralph-Inspired)

The `CompletionGate` system allows tasks to signal completion status:

```rust
// From daemon/task/completion_gate.rs
pub struct CompletionGate {
    pub ready: bool,
    pub reason: Option<String>,
    pub progress: Option<String>,
    pub blockers: Option<Vec<String>>,
}
```

**Format in Claude output:**
```xml
<COMPLETION_GATE>
ready: true
reason: "All tests pass"
progress: "Implemented feature X"
</COMPLETION_GATE>
```

### 3.2 Circuit Breaker

Prevents infinite loops in autonomous mode:

```rust
pub struct CircuitBreaker {
    pub runs_without_changes: u32,      // Trips after 3 runs with no changes
    pub same_error_count: u32,          // Trips after 5 identical errors
    pub iteration_count: u32,           // Trips after 10 iterations
    pub is_open: bool,
    pub open_reason: Option<String>,
}
```

### 3.3 Autonomous Loop Mode

When `autonomous_loop: true` on a contract:
1. Task runs to completion
2. Output is parsed for `COMPLETION_GATE`
3. If `ready: false`, task is restarted with `--continue`
4. Circuit breaker monitors for stuck states

### 3.4 Supervisor State Persistence

```rust
pub struct SupervisorState {
    pub conversation_history: serde_json::Value,
    pub pending_task_ids: Vec<Uuid>,
    pub phase: String,
    pub last_activity: DateTime<Utc>,
}
```

### 3.5 Conversation Snapshots

```rust
pub struct ConversationSnapshot {
    pub task_id: Uuid,
    pub checkpoint_id: Option<Uuid>,
    pub snapshot_type: String,  // 'auto', 'manual', 'checkpoint'
    pub conversation_state: serde_json::Value,
}
```

### 3.6 Phase Guidance System

```rust
// From llm/phase_guidance.rs
pub struct PhaseDeliverables {
    pub phase: String,
    pub recommended_files: Vec<RecommendedFile>,
    pub requires_repository: bool,
    pub requires_tasks: bool,
    pub guidance: String,
}
```

---

## 4. Extension Points for Ralph-Inspired Features

### 4.1 Task Manager Hook Points

Location: `daemon/task/manager.rs`

| Hook Point | Current State | Extension Opportunity |
|------------|---------------|----------------------|
| `spawn_task()` | Creates worktree, spawns process | Add pre-flight checks, memory injection |
| `handle_output()` | Streams to server | Enhanced context extraction |
| `on_completion()` | Cleanup, status update | Post-task analysis, learning |
| `restart_with_continue()` | Autonomous loop restart | Context summarization |

### 4.2 Process Manager Hook Points

Location: `daemon/process/claude.rs`

| Hook Point | Current State | Extension Opportunity |
|------------|---------------|----------------------|
| `build_command()` | Constructs CLI args | Dynamic prompt injection |
| `inject_system_prompt()` | Static prompts | Context-aware prompting |
| `parse_output()` | JSON message parsing | Structured output extraction |

### 4.3 Server API Extension Points

Location: `server/handlers/`

| Endpoint Category | Files | Extension Opportunity |
|-------------------|-------|----------------------|
| Contract Daemon API | `contract_daemon.rs` | Enhanced progress tracking |
| Supervisor API | `mesh_supervisor.rs` | Smarter task scheduling |
| History API | `history.rs` | Learning from past sessions |

### 4.4 Configuration Extension Points

Location: `daemon/config.rs`

```rust
pub struct ProcessConfig {
    pub claude_command: String,
    pub claude_args: Vec<String>,
    pub env_vars: HashMap<String, String>,
    // Extension: Add ralph-style configs
    // pub context_window_size: usize,
    // pub memory_extraction_enabled: bool,
    // pub learning_mode: LearningMode,
}
```

---

## 5. Current Limitations That Ralph Patterns Could Address

### 5.1 Context Window Management

**Current State:**
- No explicit context window tracking
- Conversation history stored but not summarized
- `--continue` flag relies on Claude's session state

**Ralph Pattern Opportunities:**
- Token counting per message
- Automatic context summarization when approaching limits
- Smart context pruning strategies

### 5.2 Memory and Learning

**Current State:**
- Output stored in `task_events` table
- Checkpoints stored with diffs
- No cross-task learning

**Ralph Pattern Opportunities:**
- Extract patterns from successful completions
- Build knowledge base from task outputs
- Learn from failures to improve future prompts

### 5.3 Progress Tracking

**Current State:**
- `progress_summary` field on tasks
- `COMPLETION_GATE` for completion signaling
- Phase checklist for deliverables

**Ralph Pattern Opportunities:**
- Structured progress metrics extraction
- Confidence scoring
- Automatic milestone detection

### 5.4 Error Recovery

**Current State:**
- Circuit breaker stops on repeated errors
- Daemon failover with retry count
- Manual intervention required for stuck tasks

**Ralph Pattern Opportunities:**
- Intelligent error classification
- Automatic recovery strategies
- Error pattern learning

### 5.5 Task Planning

**Current State:**
- Human-written plans
- LLM-generated task breakdowns (via `task_output.rs`)
- Static orchestrator prompts

**Ralph Pattern Opportunities:**
- Dynamic plan refinement
- Dependency inference
- Resource estimation

---

## 6. Data Flow Diagrams

### 6.1 Task Execution Flow

```
┌─────────┐    SpawnTask     ┌──────────┐
│ Server  │ ───────────────► │  Daemon  │
└─────────┘                  └────┬─────┘
                                  │
                        ┌─────────▼─────────┐
                        │   Task Manager    │
                        │   - Create worktree│
                        │   - Setup env vars │
                        └─────────┬─────────┘
                                  │
                        ┌─────────▼─────────┐
                        │  Process Manager  │
                        │   - Spawn Claude  │
                        │   - Inject prompt │
                        └─────────┬─────────┘
                                  │
                        ┌─────────▼─────────┐
                        │   Claude Code     │
                        │   - Execute task  │
                        │   - Stream output │
                        └─────────┬─────────┘
                                  │
              ┌───────────────────┼───────────────────┐
              ▼                   ▼                   ▼
       ┌────────────┐    ┌──────────────┐    ┌─────────────┐
       │ TaskOutput │    │ Checkpoints  │    │ Completion  │
       │  Events    │    │   (Git)      │    │   Gate      │
       └────────────┘    └──────────────┘    └─────────────┘
```

### 6.2 Autonomous Loop Flow

```
┌─────────────────────────────────────────────────────────┐
│                    AUTONOMOUS LOOP                       │
└─────────────────────────────────────────────────────────┘
                            │
                ┌───────────▼───────────┐
                │    Execute Task       │
                └───────────┬───────────┘
                            │
                ┌───────────▼───────────┐
                │  Parse COMPLETION_GATE │
                └───────────┬───────────┘
                            │
            ┌───────────────┼───────────────┐
            │               │               │
            ▼               ▼               ▼
      ready: true    ready: false    No Gate Found
            │               │               │
            ▼               ▼               ▼
        Complete    ┌───────────┐    ┌───────────┐
                    │  Check    │    │  Circuit  │
                    │  Circuit  │    │  Breaker  │
                    │  Breaker  │    │   Trip?   │
                    └─────┬─────┘    └─────┬─────┘
                          │                │
                   ┌──────┴──────┐  ┌──────┴──────┐
                   ▼             ▼  ▼             ▼
               Continue       Stop          Continue
              with Loop                    with Loop
```

---

## 7. Key Files Reference

### Daemon Core
- `src/daemon/mod.rs` - Module exports
- `src/daemon/task/manager.rs` - Task lifecycle (~1200 lines)
- `src/daemon/task/state.rs` - State machine
- `src/daemon/task/completion_gate.rs` - Ralph-style completion
- `src/daemon/process/claude.rs` - Claude subprocess management
- `src/daemon/worktree/manager.rs` - Git worktree isolation
- `src/daemon/ws/protocol.rs` - Server-daemon protocol

### Server Core
- `src/server/mod.rs` - Route configuration
- `src/server/handlers/mesh_supervisor.rs` - Supervisor operations
- `src/server/handlers/contract_daemon.rs` - Contract CLI interface
- `src/server/handlers/history.rs` - Conversation history

### LLM Integration
- `src/llm/phase_guidance.rs` - Phase deliverables
- `src/llm/contract_tools.rs` - Contract interaction tools
- `src/llm/task_output.rs` - Output analysis

### CLI
- `src/bin/makima.rs` - CLI entry point
- `src/daemon/cli/mod.rs` - CLI commands

---

## 8. Recommendations for Ralph Integration

### Priority 1: Enhanced Context Management
- Add token counting to message parsing
- Implement context summarization triggers
- Create memory extraction pipeline

### Priority 2: Structured Progress Tracking
- Extend `COMPLETION_GATE` format
- Add confidence scoring
- Implement milestone detection

### Priority 3: Learning System
- Create patterns database
- Implement cross-task knowledge sharing
- Add success/failure analysis

### Priority 4: Improved Error Recovery
- Classify error types
- Create recovery playbooks
- Implement automatic retry strategies

---

## Appendix: Related Configuration

### Daemon Configuration (`makima-daemon.toml`)
```toml
[server]
url = "wss://api.makima.jp"
api_key = "..."

[process]
max_concurrent_tasks = 4
claude_command = "claude"
heartbeat_commit_interval_secs = 300

[worktree]
base_dir = "~/.makima/worktrees"
repos_dir = "~/.makima/repos"
```

### Environment Variables
- `MAKIMA_API_KEY` - API authentication
- `MAKIMA_DAEMON_SERVER_URL` - Server WebSocket URL
- `MAKIMA_TASK_ID` - Set in task environment
- `MAKIMA_CONTRACT_ID` - Set in task environment