soryu/makima/src/server/handlers/mod.rs, branch makima/discuss-contract-feature

fix: Resolve merge conflict in server/mod.rs

2026-02-03T23:19:40+00:00

Combine imports from both branches - include both chains and contract_discuss handlers. Co-Authored-By: Claude Opus 4.5

Add makima chain mechanism

2026-02-03T22:01:37+00:00

[WIP] Heartbeat checkpoint - 2026-02-03 18:21:43 UTC

2026-02-03T18:21:43+00:00

Make makima more opinionated and structured

2026-02-02T02:34:50+00:00

Replace TTS endpoint with Rust-native Qwen3-TTS (#41)

2026-01-28T03:50:45+00:00

* chore: fix unused import warnings in qwen3-tts module - Remove unused import 'IndexOp' in model.rs - Remove unused import 'DType' in speech_tokenizer.rs - Add #[allow(dead_code)] to codebook_dim field in RvqCodebook Co-Authored-By: Claude Opus 4.5 * feat: add voice loading and selection for TTS cloning Add voice reference audio loading so the TTS speak handler can perform voice cloning using reference WAV files from the voices/ directory. - Add voice.rs module: loads manifest.json and reference.wav for a given voice_id, decodes via symphonia, resamples to 24kHz for the TTS engine - Update speak.rs: resolve voice_id from the speak request (default "makima"), load reference audio, pass it to engine.generate() - Add voices/makima/README.md with instructions for obtaining reference audio (extraction from YouTube, recording, ffmpeg conversion) - Graceful fallback: if reference audio is missing, TTS proceeds without voice cloning using the model's default voice Co-Authored-By: Claude Opus 4.5 * [WIP] Heartbeat checkpoint - 2026-01-28 03:49:13 UTC --------- Co-authored-by: Claude Opus 4.5

Add Qwen3-TTS streaming endpoint for voice synthesis (#40)

2026-01-28T02:54:17+00:00

* Task completion checkpoint * Task completion checkpoint * Task completion checkpoint * Add Qwen3-TTS research document for live TTS replacement Research findings for replacing Chatterbox TTS with Qwen3-TTS-12Hz-0.6B-Base: - Current TTS: Chatterbox-Turbo-ONNX with batch-only generation, no streaming - Qwen3-TTS: 97ms end-to-end latency, streaming support, 3-second voice cloning - Voice cloning: Requires 3s reference audio + transcript (Makima voice planned) - Integration: Python service with WebSocket bridge (no ONNX export available) - Languages: 10 supported including English and Japanese Document includes: - Current architecture analysis (makima/src/tts.rs) - Qwen3-TTS capabilities and requirements - Feasibility assessment for live/streaming TTS - Audio clip requirements for voice cloning - Preliminary technical approach with architecture diagrams Co-Authored-By: Claude Opus 4.5 * [WIP] Heartbeat checkpoint - 2026-01-27 03:11:15 UTC * Add Qwen3-TTS research documentation Comprehensive research on replacing Chatterbox TTS with Qwen3-TTS-12Hz-0.6B-Base: - Current TTS implementation analysis (Chatterbox-Turbo-ONNX in makima/src/tts.rs) - Qwen3-TTS capabilities: 97ms streaming latency, voice cloning with 3s reference - Cross-lingual support: Japanese voice (Makima/Tomori Kusunoki) speaking English - Python microservice architecture recommendation (FastAPI + WebSocket) - Implementation phases and technical approach - Hardware requirements and dependencies Key findings: - Live/streaming TTS is highly feasible with 97ms latency - Voice cloning fully supported with 0.95 speaker similarity - Recommended: Python microservice with WebSocket streaming Co-Authored-By: Claude Opus 4.5 * Add comprehensive Qwen3-TTS integration specification This specification document defines the complete integration of Qwen3-TTS-12Hz-0.6B-Base as a replacement for the existing Chatterbox-Turbo TTS implementation. The document covers: ## Functional Requirements - WebSocket endpoint /api/v1/speak for streaming TTS - Voice cloning with default Makima voice (Japanese VA speaking English) - Support for custom voice references - Detailed client-to-server and server-to-client message protocols - Integration with Listen page for bidirectional speech ## Non-Functional Requirements - Latency targets: < 200ms first audio byte - Audio quality: 24kHz, mono, PCM16/PCM32f - Hardware requirements: CUDA GPU with 4-8GB VRAM - Scalability: 10 concurrent sessions per GPU ## Architecture Specification - Python TTS microservice with FastAPI/WebSocket - Rust proxy endpoint in makima server - Voice prompt caching mechanism (LRU cache) - Error handling and recovery strategies ## API Contract - Complete WebSocket message format definitions (TypeScript) - Error codes and responses (TTS_UNAVAILABLE, SYNTHESIS_ERROR, etc.) - Session state machine and lifecycle management ## Voice Asset Requirements - Makima voice clip specifications (5-10s WAV, transcript required) - Storage location: models/voices/makima/ - Metadata format for voice management ## Testing Strategy - Unit tests for Python TTS service and Rust proxy - Integration tests for WebSocket flow - Latency benchmarks with performance targets - Test data fixtures for various text lengths Co-Authored-By: Claude Opus 4.5 * Add Qwen3-TTS implementation plan Comprehensive implementation plan for replacing Chatterbox-TTS with Qwen3-TTS streaming TTS service, including: - Task breakdown with estimated hours for each phase - Phase 1: Python TTS microservice (FastAPI, WebSocket) - Phase 2: Rust proxy integration (speak.rs, tts_client.rs) - Detailed file changes and new module structure - Testing plan with unit, integration, and latency benchmarks - Risk assessment with mitigation strategies - Success criteria for each phase Based on specification in docs/specs/qwen3-tts-spec.md Co-Authored-By: Claude Opus 4.5 * Add author and research references to TTS implementation plan Add links to research documentation and author attribution. Co-Authored-By: Claude Opus 4.5 * [WIP] Heartbeat checkpoint - 2026-01-27 03:25:06 UTC * Add Python TTS service project structure (Phase 1.1-1.3) Create the initial makima-tts Python service directory structure with: - pyproject.toml with FastAPI, Qwen-TTS, and torch dependencies - config.py with pydantic-settings TTSConfig class - models.py with Pydantic message models (Start, Speak, Stop, Ready, etc.) This implements tasks P1.1, P1.2, and P1.3 from the Qwen3-TTS implementation plan. Co-Authored-By: Claude Opus 4.5 * Add TTS engine and voice manager for Qwen3-TTS (Phase 1.4-1.5) Implement core TTS functionality: - tts_engine.py: Qwen3-TTS wrapper with streaming audio chunk generation - voice_manager.py: Voice prompt caching with LRU eviction and TTL support Co-Authored-By: Claude Opus 4.5 * [WIP] Heartbeat checkpoint - 2026-01-27 03:30:06 UTC * Add TTS proxy client and message types (Phase 2.1, 2.2, 2.4) - Add tts_client.rs with TtsConfig, TtsCircuitBreaker, TtsError, TtsProxyClient, and TtsConnection structs for WebSocket proxying - Add TTS message types to messages.rs (TtsAudioEncoding, TtsPriority, TtsStartMessage, TtsSpeakMessage, TtsStopMessage, TtsClientMessage, TtsReadyMessage, TtsAudioChunkMessage, TtsCompleteMessage, TtsErrorMessage, TtsStoppedMessage, TtsServerMessage) - Export tts_client module from server mod.rs - tokio-tungstenite already present in Cargo.toml Co-Authored-By: Claude Opus 4.5 * Add TTS WebSocket handler and route (Phase 2.3, 2.5, 2.6) - Create speak.rs WebSocket handler that proxies to Python TTS service - Add TtsState fields (tts_client, tts_config) to AppState - Add with_tts() builder and is_tts_healthy() methods to AppState - Register /api/v1/speak route in the router - Add speak module export in handlers/mod.rs The handler forwards WebSocket messages bidirectionally between the client and the Python TTS microservice with proper error handling. Co-Authored-By: Claude Opus 4.5 * Add Makima voice profile assets for TTS voice cloning Creates the voice assets directory structure with: - manifest.json containing voice configuration (voice_id, speaker, language, reference audio path, and Japanese transcript placeholder) - README.md with instructions for obtaining voice reference audio Co-Authored-By: Claude Opus 4.5 * Add Rust-native Qwen3-TTS integration research document Research findings for integrating Qwen3-TTS-12Hz-0.6B-Base directly into the makima Rust codebase without Python. Key conclusions: - ONNX export is not viable (unsupported architecture) - Candle (HF Rust ML framework) is the recommended approach - Model weights available in safetensors format (2.52GB total) - Three components needed: LM backbone, code predictor, speech tokenizer - Crane project has Qwen3-TTS as highest priority (potential upstream) Co-Authored-By: Claude Opus 4.5 * [WIP] Heartbeat checkpoint - 2026-01-27 11:21:43 UTC * [WIP] Heartbeat checkpoint - 2026-01-27 11:24:19 UTC * [WIP] Heartbeat checkpoint - 2026-01-27 11:26:43 UTC * feat: implement Rust-native Qwen3-TTS using candle framework Replace monolithic tts.rs with modular tts/ directory structure: - tts/mod.rs: TtsEngine trait, TtsEngineFactory, shared types (AudioChunk, TtsError), and utility functions (save_wav, resample, argmax) - tts/chatterbox.rs: existing ONNX-based ChatterboxTTS adapted to implement TtsEngine trait with Mutex-wrapped sessions for Send+Sync - tts/qwen3/mod.rs: Qwen3Tts entry point with HuggingFace model loading - tts/qwen3/config.rs: Qwen3TtsConfig parsing from HF config.json - tts/qwen3/model.rs: 28-layer Qwen3 transformer with RoPE, GQA (16 heads, 8 KV heads), SiLU MLP, RMS norm, and KV cache - tts/qwen3/code_predictor.rs: 5-layer MTP module predicting 16 codebooks - tts/qwen3/speech_tokenizer.rs: ConvNet encoder/decoder with 16-layer RVQ - tts/qwen3/generate.rs: autoregressive generation loop with streaming support Add candle-core, candle-nn, candle-transformers, safetensors to Cargo.toml. Co-Authored-By: Claude Opus 4.5 * feat: integrate TTS engine into speak WebSocket handler - Update speak.rs handler to use TTS engine directly from SharedState instead of returning a stub "not implemented" error - Add TtsEngine (OnceCell lazy-loaded) to AppState in state.rs with get_tts_engine() method for lazy initialization on first connection - Implement full WebSocket protocol: client sends JSON speak/cancel/stop messages, server streams binary PCM audio chunks and audio_end signals - Create voices/makima/manifest.json for Makima voice profile configuration - All files compile successfully with zero errors Co-Authored-By: Claude Opus 4.5 * feat: add /speak TTS page with WebSocket audio playback Add a new /speak frontend page for text-to-speech via WebSocket. The page accepts text input and streams synthesized PCM audio through the Web Audio API. Includes model loading indicator, cancel support, and connection status. Also adds a loading bar to the listen page ControlPanel during WebSocket connection. Co-Authored-By: Claude Opus 4.5 --------- Co-authored-by: Claude Opus 4.5

Add Red Team adversarial review system for contract monitoring (#35)

2026-01-27T01:05:25+00:00

Implements a parallel "red team" task that monitors work task outputs in real-time, verifying implementations stick to contract requirements, repository standards, and the execution plan. Key features: - New `red_team_enabled` and `red_team_prompt` contract configuration - Red team tasks auto-spawn when first work task is created - `makima red-team notify` CLI command for alerting supervisors - POST /api/v1/mesh/red-team/notify and /status endpoints - Alert delivery to supervisor via SendMessage daemon command - Notification audit trail via history_events table Database changes: - Add red_team_enabled/red_team_prompt columns to contracts - Add is_red_team flag to tasks with partial index - Create red_team_notifications table for audit logging Co-authored-by: Claude Opus 4.5

Add resume and history system for makima (#1)

2026-01-15T22:33:47+00:00

This PR implements a comprehensive resume and history system that enables: 1. **History Viewing** - View complete conversation history for contracts across all phases - View conversation history for individual tasks - View task output/tool call history with timestamps - View checkpoint history - Timeline view showing all activities 2. **Resume System** - Resume interrupted supervisor conversations with full context - Resume interrupted task conversations - Resume from specific checkpoints - Continue tasks from previous task state (worktree inheritance) 3. **Rewind/Restore Features** - Rewind code to any checkpoint (git restore) - Rewind conversation to any point - Create new branches from historical points - Fork tasks from any point in history - New migration: 20250117000000_history_tables.sql - conversation_snapshots table for storing conversation state - history_events table for unified timeline - Added forking fields to tasks table - Added conversation_snapshot_id to task_checkpoints - ConversationSnapshot, HistoryEvent, ConversationMessage - Request/response types for resume and rewind operations - Query filter types for history endpoints - CRUD functions for conversation_snapshots - CRUD functions for history_events - Task conversation retrieval from task_events - GET /api/v1/contracts/{id}/history - GET /api/v1/contracts/{id}/supervisor/conversation - GET /api/v1/mesh/tasks/{id}/conversation - GET /api/v1/timeline - POST /api/v1/contracts/{id}/supervisor/resume - POST /api/v1/mesh/tasks/{id}/rewind - POST /api/v1/mesh/tasks/{id}/fork - POST /api/v1/mesh/tasks/{id}/checkpoints/{cid}/resume - POST /api/v1/mesh/tasks/{id}/checkpoints/{cid}/branch - POST /api/v1/contracts/{id}/supervisor/conversation/rewind - task-history: View task conversation history - task-checkpoints: List task checkpoints - resume: Resume supervisor after interruption - task-resume-from: Resume task from checkpoint - task-rewind: Rewind task code to checkpoint - task-fork: Fork task from historical point - rewind-conversation: Rewind supervisor conversation

Add repository history feature to store and suggest previously used repositories (#18)

2026-01-15T03:37:44+00:00

- Add repository_history table migration with repo_type, repo_path, use_count, last_used_at - Add RepositoryHistoryEntry model and CRUD database functions - Create API endpoints: GET/POST/DELETE /api/v1/repository-history, GET /api/v1/repository-history/suggestions - Update add_remote_repository and add_local_repository handlers to automatically track history - Update frontend API with repository history types and functions - Add Repository History section to Settings page with list of entries and delete functionality - Add suggestions dropdown to RepositoryPanel when entering new repository URL/path - Suggestions filter by repo type (remote vs local) and match on user input Test plan: - Add a remote repository to a contract - verify it appears in Settings history - Add a local repository to a contract - verify it appears in Settings history - Add same repository again - verify use_count increments, not duplicate - When adding new repository, verify suggestions appear based on history - Delete a history entry from Settings - verify it's removed - Verify suggestions only show matching type (remote for remote, local for local) Co-authored-by: Claude Opus 4.5

feat(listen): add transcript analysis API endpoints

2026-01-15T01:30:02+00:00

Adds three new endpoints under /api/v1/listen/: - POST /analyze - Analyze a file's transcript for requirements, decisions, action items - POST /create-contract - Create a new contract from analyzed transcript - POST /update-contract - Update an existing contract with extracted information These endpoints bridge the Listen (voice transcription) system with the Contract system, enabling voice-to-contract workflows. Co-Authored-By: Claude Opus 4.5