| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The contracts table, supervisor task type, and all their backing
machinery have been inert for several PRs. The directives system reads
its own active contract body for spec text, and PR #135 removed the
last LLM surface that spawned supervisors.
This PR wipes the dead surface in one shot — the user authorised a DB
wipe, so the migration drops every legacy table with CASCADE rather
than carrying forward stub rows. Net change: −12k LOC across handlers,
repository, state, models, the TUI, and the listen module.
What's gone:
- contracts, contract_chat_*, contract_events, contract_repositories,
contract_type_templates tables.
- supervisor_states, supervisor_heartbeats tables.
- mesh_chat_conversations, mesh_chat_messages tables.
- tasks.contract_id/is_supervisor/supervisor_task_id/supervisor_worktree_task_id columns.
- directive_steps.contract_id/contract_type columns.
- files.contract_id/contract_phase columns.
- history_events.contract_id/phase columns.
- The Contract/Supervisor/MeshChat handler + model + repository
surface, plus the daemon TUI views that read them.
- The standalone listen.rs websocket handler (orphaned with the LLM).
What stays:
- mesh_supervisor handler: trimmed to just the questions + orders
backchannel used by `makima directive ask` / `create-order` (kept
the URL prefix for CLI client compat).
- directive_documents (the user-facing "contracts" surface).
- pending_questions in-memory state for the directive Ask flow.
cargo check, cargo test --lib (68 passed), tsc, and vite build all
clean.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Task completion checkpoint
* Task completion checkpoint
* Task completion checkpoint
* Add Qwen3-TTS research document for live TTS replacement
Research findings for replacing Chatterbox TTS with Qwen3-TTS-12Hz-0.6B-Base:
- Current TTS: Chatterbox-Turbo-ONNX with batch-only generation, no streaming
- Qwen3-TTS: 97ms end-to-end latency, streaming support, 3-second voice cloning
- Voice cloning: Requires 3s reference audio + transcript (Makima voice planned)
- Integration: Python service with WebSocket bridge (no ONNX export available)
- Languages: 10 supported including English and Japanese
Document includes:
- Current architecture analysis (makima/src/tts.rs)
- Qwen3-TTS capabilities and requirements
- Feasibility assessment for live/streaming TTS
- Audio clip requirements for voice cloning
- Preliminary technical approach with architecture diagrams
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* [WIP] Heartbeat checkpoint - 2026-01-27 03:11:15 UTC
* Add Qwen3-TTS research documentation
Comprehensive research on replacing Chatterbox TTS with Qwen3-TTS-12Hz-0.6B-Base:
- Current TTS implementation analysis (Chatterbox-Turbo-ONNX in makima/src/tts.rs)
- Qwen3-TTS capabilities: 97ms streaming latency, voice cloning with 3s reference
- Cross-lingual support: Japanese voice (Makima/Tomori Kusunoki) speaking English
- Python microservice architecture recommendation (FastAPI + WebSocket)
- Implementation phases and technical approach
- Hardware requirements and dependencies
Key findings:
- Live/streaming TTS is highly feasible with 97ms latency
- Voice cloning fully supported with 0.95 speaker similarity
- Recommended: Python microservice with WebSocket streaming
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add comprehensive Qwen3-TTS integration specification
This specification document defines the complete integration of
Qwen3-TTS-12Hz-0.6B-Base as a replacement for the existing Chatterbox-Turbo
TTS implementation. The document covers:
## Functional Requirements
- WebSocket endpoint /api/v1/speak for streaming TTS
- Voice cloning with default Makima voice (Japanese VA speaking English)
- Support for custom voice references
- Detailed client-to-server and server-to-client message protocols
- Integration with Listen page for bidirectional speech
## Non-Functional Requirements
- Latency targets: < 200ms first audio byte
- Audio quality: 24kHz, mono, PCM16/PCM32f
- Hardware requirements: CUDA GPU with 4-8GB VRAM
- Scalability: 10 concurrent sessions per GPU
## Architecture Specification
- Python TTS microservice with FastAPI/WebSocket
- Rust proxy endpoint in makima server
- Voice prompt caching mechanism (LRU cache)
- Error handling and recovery strategies
## API Contract
- Complete WebSocket message format definitions (TypeScript)
- Error codes and responses (TTS_UNAVAILABLE, SYNTHESIS_ERROR, etc.)
- Session state machine and lifecycle management
## Voice Asset Requirements
- Makima voice clip specifications (5-10s WAV, transcript required)
- Storage location: models/voices/makima/
- Metadata format for voice management
## Testing Strategy
- Unit tests for Python TTS service and Rust proxy
- Integration tests for WebSocket flow
- Latency benchmarks with performance targets
- Test data fixtures for various text lengths
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add Qwen3-TTS implementation plan
Comprehensive implementation plan for replacing Chatterbox-TTS with
Qwen3-TTS streaming TTS service, including:
- Task breakdown with estimated hours for each phase
- Phase 1: Python TTS microservice (FastAPI, WebSocket)
- Phase 2: Rust proxy integration (speak.rs, tts_client.rs)
- Detailed file changes and new module structure
- Testing plan with unit, integration, and latency benchmarks
- Risk assessment with mitigation strategies
- Success criteria for each phase
Based on specification in docs/specs/qwen3-tts-spec.md
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add author and research references to TTS implementation plan
Add links to research documentation and author attribution.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* [WIP] Heartbeat checkpoint - 2026-01-27 03:25:06 UTC
* Add Python TTS service project structure (Phase 1.1-1.3)
Create the initial makima-tts Python service directory structure with:
- pyproject.toml with FastAPI, Qwen-TTS, and torch dependencies
- config.py with pydantic-settings TTSConfig class
- models.py with Pydantic message models (Start, Speak, Stop, Ready, etc.)
This implements tasks P1.1, P1.2, and P1.3 from the Qwen3-TTS implementation plan.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add TTS engine and voice manager for Qwen3-TTS (Phase 1.4-1.5)
Implement core TTS functionality:
- tts_engine.py: Qwen3-TTS wrapper with streaming audio chunk generation
- voice_manager.py: Voice prompt caching with LRU eviction and TTL support
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* [WIP] Heartbeat checkpoint - 2026-01-27 03:30:06 UTC
* Add TTS proxy client and message types (Phase 2.1, 2.2, 2.4)
- Add tts_client.rs with TtsConfig, TtsCircuitBreaker, TtsError,
TtsProxyClient, and TtsConnection structs for WebSocket proxying
- Add TTS message types to messages.rs (TtsAudioEncoding, TtsPriority,
TtsStartMessage, TtsSpeakMessage, TtsStopMessage, TtsClientMessage,
TtsReadyMessage, TtsAudioChunkMessage, TtsCompleteMessage,
TtsErrorMessage, TtsStoppedMessage, TtsServerMessage)
- Export tts_client module from server mod.rs
- tokio-tungstenite already present in Cargo.toml
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add TTS WebSocket handler and route (Phase 2.3, 2.5, 2.6)
- Create speak.rs WebSocket handler that proxies to Python TTS service
- Add TtsState fields (tts_client, tts_config) to AppState
- Add with_tts() builder and is_tts_healthy() methods to AppState
- Register /api/v1/speak route in the router
- Add speak module export in handlers/mod.rs
The handler forwards WebSocket messages bidirectionally between
the client and the Python TTS microservice with proper error handling.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add Makima voice profile assets for TTS voice cloning
Creates the voice assets directory structure with:
- manifest.json containing voice configuration (voice_id, speaker,
language, reference audio path, and Japanese transcript placeholder)
- README.md with instructions for obtaining voice reference audio
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add Rust-native Qwen3-TTS integration research document
Research findings for integrating Qwen3-TTS-12Hz-0.6B-Base directly into
the makima Rust codebase without Python. Key conclusions:
- ONNX export is not viable (unsupported architecture)
- Candle (HF Rust ML framework) is the recommended approach
- Model weights available in safetensors format (2.52GB total)
- Three components needed: LM backbone, code predictor, speech tokenizer
- Crane project has Qwen3-TTS as highest priority (potential upstream)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* [WIP] Heartbeat checkpoint - 2026-01-27 11:21:43 UTC
* [WIP] Heartbeat checkpoint - 2026-01-27 11:24:19 UTC
* [WIP] Heartbeat checkpoint - 2026-01-27 11:26:43 UTC
* feat: implement Rust-native Qwen3-TTS using candle framework
Replace monolithic tts.rs with modular tts/ directory structure:
- tts/mod.rs: TtsEngine trait, TtsEngineFactory, shared types (AudioChunk,
TtsError), and utility functions (save_wav, resample, argmax)
- tts/chatterbox.rs: existing ONNX-based ChatterboxTTS adapted to implement
TtsEngine trait with Mutex-wrapped sessions for Send+Sync
- tts/qwen3/mod.rs: Qwen3Tts entry point with HuggingFace model loading
- tts/qwen3/config.rs: Qwen3TtsConfig parsing from HF config.json
- tts/qwen3/model.rs: 28-layer Qwen3 transformer with RoPE, GQA (16 heads,
8 KV heads), SiLU MLP, RMS norm, and KV cache
- tts/qwen3/code_predictor.rs: 5-layer MTP module predicting 16 codebooks
- tts/qwen3/speech_tokenizer.rs: ConvNet encoder/decoder with 16-layer RVQ
- tts/qwen3/generate.rs: autoregressive generation loop with streaming support
Add candle-core, candle-nn, candle-transformers, safetensors to Cargo.toml.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: integrate TTS engine into speak WebSocket handler
- Update speak.rs handler to use TTS engine directly from SharedState
instead of returning a stub "not implemented" error
- Add TtsEngine (OnceCell lazy-loaded) to AppState in state.rs with
get_tts_engine() method for lazy initialization on first connection
- Implement full WebSocket protocol: client sends JSON speak/cancel/stop
messages, server streams binary PCM audio chunks and audio_end signals
- Create voices/makima/manifest.json for Makima voice profile configuration
- All files compile successfully with zero errors
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: add /speak TTS page with WebSocket audio playback
Add a new /speak frontend page for text-to-speech via WebSocket.
The page accepts text input and streams synthesized PCM audio through
the Web Audio API. Includes model loading indicator, cancel support,
and connection status. Also adds a loading bar to the listen page
ControlPanel during WebSocket connection.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
|
