Add Qwen3-TTS streaming endpoint for voice synthesis (#40)

* Task completion checkpoint

* Task completion checkpoint

* Task completion checkpoint

* Add Qwen3-TTS research document for live TTS replacement

Research findings for replacing Chatterbox TTS with Qwen3-TTS-12Hz-0.6B-Base:

- Current TTS: Chatterbox-Turbo-ONNX with batch-only generation, no streaming
- Qwen3-TTS: 97ms end-to-end latency, streaming support, 3-second voice cloning
- Voice cloning: Requires 3s reference audio + transcript (Makima voice planned)
- Integration: Python service with WebSocket bridge (no ONNX export available)
- Languages: 10 supported including English and Japanese

Document includes:
- Current architecture analysis (makima/src/tts.rs)
- Qwen3-TTS capabilities and requirements
- Feasibility assessment for live/streaming TTS
- Audio clip requirements for voice cloning
- Preliminary technical approach with architecture diagrams

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* [WIP] Heartbeat checkpoint - 2026-01-27 03:11:15 UTC

* Add Qwen3-TTS research documentation

Comprehensive research on replacing Chatterbox TTS with Qwen3-TTS-12Hz-0.6B-Base:

- Current TTS implementation analysis (Chatterbox-Turbo-ONNX in makima/src/tts.rs)
- Qwen3-TTS capabilities: 97ms streaming latency, voice cloning with 3s reference
- Cross-lingual support: Japanese voice (Makima/Tomori Kusunoki) speaking English
- Python microservice architecture recommendation (FastAPI + WebSocket)
- Implementation phases and technical approach
- Hardware requirements and dependencies

Key findings:
- Live/streaming TTS is highly feasible with 97ms latency
- Voice cloning fully supported with 0.95 speaker similarity
- Recommended: Python microservice with WebSocket streaming

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add comprehensive Qwen3-TTS integration specification

This specification document defines the complete integration of
Qwen3-TTS-12Hz-0.6B-Base as a replacement for the existing Chatterbox-Turbo
TTS implementation. The document covers:

## Functional Requirements
- WebSocket endpoint /api/v1/speak for streaming TTS
- Voice cloning with default Makima voice (Japanese VA speaking English)
- Support for custom voice references
- Detailed client-to-server and server-to-client message protocols
- Integration with Listen page for bidirectional speech

## Non-Functional Requirements
- Latency targets: < 200ms first audio byte
- Audio quality: 24kHz, mono, PCM16/PCM32f
- Hardware requirements: CUDA GPU with 4-8GB VRAM
- Scalability: 10 concurrent sessions per GPU

## Architecture Specification
- Python TTS microservice with FastAPI/WebSocket
- Rust proxy endpoint in makima server
- Voice prompt caching mechanism (LRU cache)
- Error handling and recovery strategies

## API Contract
- Complete WebSocket message format definitions (TypeScript)
- Error codes and responses (TTS_UNAVAILABLE, SYNTHESIS_ERROR, etc.)
- Session state machine and lifecycle management

## Voice Asset Requirements
- Makima voice clip specifications (5-10s WAV, transcript required)
- Storage location: models/voices/makima/
- Metadata format for voice management

## Testing Strategy
- Unit tests for Python TTS service and Rust proxy
- Integration tests for WebSocket flow
- Latency benchmarks with performance targets
- Test data fixtures for various text lengths

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add Qwen3-TTS implementation plan

Comprehensive implementation plan for replacing Chatterbox-TTS with
Qwen3-TTS streaming TTS service, including:

- Task breakdown with estimated hours for each phase
- Phase 1: Python TTS microservice (FastAPI, WebSocket)
- Phase 2: Rust proxy integration (speak.rs, tts_client.rs)
- Detailed file changes and new module structure
- Testing plan with unit, integration, and latency benchmarks
- Risk assessment with mitigation strategies
- Success criteria for each phase

Based on specification in docs/specs/qwen3-tts-spec.md

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add author and research references to TTS implementation plan

Add links to research documentation and author attribution.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* [WIP] Heartbeat checkpoint - 2026-01-27 03:25:06 UTC

* Add Python TTS service project structure (Phase 1.1-1.3)

Create the initial makima-tts Python service directory structure with:
- pyproject.toml with FastAPI, Qwen-TTS, and torch dependencies
- config.py with pydantic-settings TTSConfig class
- models.py with Pydantic message models (Start, Speak, Stop, Ready, etc.)

This implements tasks P1.1, P1.2, and P1.3 from the Qwen3-TTS implementation plan.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add TTS engine and voice manager for Qwen3-TTS (Phase 1.4-1.5)

Implement core TTS functionality:
- tts_engine.py: Qwen3-TTS wrapper with streaming audio chunk generation
- voice_manager.py: Voice prompt caching with LRU eviction and TTL support

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* [WIP] Heartbeat checkpoint - 2026-01-27 03:30:06 UTC

* Add TTS proxy client and message types (Phase 2.1, 2.2, 2.4)

- Add tts_client.rs with TtsConfig, TtsCircuitBreaker, TtsError,
  TtsProxyClient, and TtsConnection structs for WebSocket proxying
- Add TTS message types to messages.rs (TtsAudioEncoding, TtsPriority,
  TtsStartMessage, TtsSpeakMessage, TtsStopMessage, TtsClientMessage,
  TtsReadyMessage, TtsAudioChunkMessage, TtsCompleteMessage,
  TtsErrorMessage, TtsStoppedMessage, TtsServerMessage)
- Export tts_client module from server mod.rs
- tokio-tungstenite already present in Cargo.toml

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add TTS WebSocket handler and route (Phase 2.3, 2.5, 2.6)

- Create speak.rs WebSocket handler that proxies to Python TTS service
- Add TtsState fields (tts_client, tts_config) to AppState
- Add with_tts() builder and is_tts_healthy() methods to AppState
- Register /api/v1/speak route in the router
- Add speak module export in handlers/mod.rs

The handler forwards WebSocket messages bidirectionally between
the client and the Python TTS microservice with proper error handling.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add Makima voice profile assets for TTS voice cloning

Creates the voice assets directory structure with:
- manifest.json containing voice configuration (voice_id, speaker,
  language, reference audio path, and Japanese transcript placeholder)
- README.md with instructions for obtaining voice reference audio

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add Rust-native Qwen3-TTS integration research document

Research findings for integrating Qwen3-TTS-12Hz-0.6B-Base directly into
the makima Rust codebase without Python. Key conclusions:
- ONNX export is not viable (unsupported architecture)
- Candle (HF Rust ML framework) is the recommended approach
- Model weights available in safetensors format (2.52GB total)
- Three components needed: LM backbone, code predictor, speech tokenizer
- Crane project has Qwen3-TTS as highest priority (potential upstream)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* [WIP] Heartbeat checkpoint - 2026-01-27 11:21:43 UTC

* [WIP] Heartbeat checkpoint - 2026-01-27 11:24:19 UTC

* [WIP] Heartbeat checkpoint - 2026-01-27 11:26:43 UTC

* feat: implement Rust-native Qwen3-TTS using candle framework

Replace monolithic tts.rs with modular tts/ directory structure:

- tts/mod.rs: TtsEngine trait, TtsEngineFactory, shared types (AudioChunk,
  TtsError), and utility functions (save_wav, resample, argmax)
- tts/chatterbox.rs: existing ONNX-based ChatterboxTTS adapted to implement
  TtsEngine trait with Mutex-wrapped sessions for Send+Sync
- tts/qwen3/mod.rs: Qwen3Tts entry point with HuggingFace model loading
- tts/qwen3/config.rs: Qwen3TtsConfig parsing from HF config.json
- tts/qwen3/model.rs: 28-layer Qwen3 transformer with RoPE, GQA (16 heads,
  8 KV heads), SiLU MLP, RMS norm, and KV cache
- tts/qwen3/code_predictor.rs: 5-layer MTP module predicting 16 codebooks
- tts/qwen3/speech_tokenizer.rs: ConvNet encoder/decoder with 16-layer RVQ
- tts/qwen3/generate.rs: autoregressive generation loop with streaming support

Add candle-core, candle-nn, candle-transformers, safetensors to Cargo.toml.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: integrate TTS engine into speak WebSocket handler

- Update speak.rs handler to use TTS engine directly from SharedState
  instead of returning a stub "not implemented" error
- Add TtsEngine (OnceCell lazy-loaded) to AppState in state.rs with
  get_tts_engine() method for lazy initialization on first connection
- Implement full WebSocket protocol: client sends JSON speak/cancel/stop
  messages, server streams binary PCM audio chunks and audio_end signals
- Create voices/makima/manifest.json for Makima voice profile configuration
- All files compile successfully with zero errors

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add /speak TTS page with WebSocket audio playback

Add a new /speak frontend page for text-to-speech via WebSocket.
The page accepts text input and streams synthesized PCM audio through
the Web Audio API. Includes model loading indicator, cancel support,
and connection status. Also adds a loading bar to the listen page
ControlPanel during WebSocket connection.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

1 files changed, 928 insertions, 0 deletions

diff --git a/docs/specs/qwen3-tts-spec.md b/docs/specs/qwen3-tts-spec.md
new file mode 100644
index 0000000..91d447d
--- /dev/null
+++ b/docs/specs/qwen3-tts-spec.md
@@ -0,0 +1,928 @@
+# Qwen3-TTS Integration Specification
+
+**Version:** 2.0
+**Date:** 2026-01-27
+**Status:** Draft
+**Author:** Makima Engineering
+
+## Table of Contents
+
+1. [Overview](#1-overview)
+2. [Functional Requirements](#2-functional-requirements)
+3. [Non-Functional Requirements](#3-non-functional-requirements)
+4. [Architecture Specification](#4-architecture-specification)
+5. [API Contract](#5-api-contract)
+6. [Voice Asset Requirements](#6-voice-asset-requirements)
+7. [Testing Strategy](#7-testing-strategy)
+8. [Implementation Phases](#8-implementation-phases)
+9. [Appendix](#appendix)
+
+---
+
+## 1. Overview
+
+### 1.1 Purpose
+
+This specification defines the integration of Qwen3-TTS-12Hz-0.6B-Base as a replacement for the existing Chatterbox-Turbo TTS implementation in the makima system. The new implementation is a **pure Rust** solution using the **candle** ML framework — no Python, no separate microservice. The TTS model runs directly inside the main makima process. The implementation will provide:
+
+- **Streaming TTS** with near-real-time audio synthesis
+- **Voice cloning** with default Makima voice (Japanese voice actress speaking English)
+- **Bidirectional speech integration** for the Listen page
+- **WebSocket-based streaming API** for low-latency delivery
+
+### 1.2 Background
+
+The current TTS implementation (Chatterbox-Turbo-ONNX) has limitations:
+- No streaming support (batch-only generation)
+- No HTTP/WebSocket endpoint exposed
+- High latency for interactive use cases
+
+Qwen3-TTS offers significant improvements:
+- **97ms** end-to-end latency (vs. batch processing)
+- **10 languages** supported including Japanese cross-lingual cloning
+- **3-second** reference audio for voice cloning
+- **Dual-track streaming architecture**
+
+### 1.3 Scope
+
+This specification covers:
+- WebSocket endpoint `/api/v1/speak` for streaming TTS
+- Pure Rust candle-based model inference running in-process
+- Voice asset management
+- Testing and benchmarking
+
+Out of scope:
+- ONNX export of Qwen3-TTS (not available)
+- Instruction-following TTS features (base model only)
+- Full replacement of STT/Listen functionality
+
+---
+
+## 2. Functional Requirements
+
+### 2.1 WebSocket Endpoint: `/api/v1/speak`
+
+The TTS service SHALL be exposed via a WebSocket endpoint at `/api/v1/speak` for streaming audio synthesis.
+
+#### 2.1.1 Connection Flow
+
+```
+Client                           Server (Rust/Axum)
+   |                                  |
+   |--- WS Connect ------------------>|
+   |                                  | [Load TTS model lazily if needed]
+   |<-- Ready (session_id) -----------|
+   |                                  |
+   |--- Start (config) -------------->|
+   |<-- Started ----------------------| [Load voice prompt]
+   |                                  |
+   |--- Speak (text) ---------------->| [Direct candle model inference]
+   |                                  |
+   |<-- AudioChunk (binary) ----------|
+   |<-- AudioChunk (binary) ----------|
+   |<-- Complete ---------------------|
+   |                                  |
+   |--- Stop ------------------------>|
+   |<-- Stopped ----------------------|
+   |                                  |
+```
+
+#### 2.1.2 Voice Cloning
+
+The system SHALL support voice cloning with the following modes:
+
+| Mode | Description | Requirements |
+|------|-------------|--------------|
+| Default (Makima) | Pre-loaded Makima voice | None (auto-selected) |
+| Custom Voice | User-provided reference | Audio file + transcript |
+| X-Vector Only | Speaker embedding only | Audio file (no transcript) |
+
+**Default Voice Behavior:**
+- If no voice is specified, use the pre-loaded Makima voice prompt
+- Makima voice SHALL be a Japanese voice actress (Tomori Kusunoki) speaking English
+- Voice prompt is pre-computed at model load time for zero-latency switching
+
+#### 2.1.3 Message Protocol
+
+##### Client-to-Server Messages
+
+All messages use JSON format with a `type` field for routing.
+
+**Start Message** - Initialize TTS session
+```json
+{
+  "type": "start",
+  "sampleRate": 24000,
+  "encoding": "pcm16",
+  "voice": "makima",
+  "language": "English",
+  "authToken": "optional-jwt-token",
+  "contractId": "optional-contract-uuid"
+}
+```
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `type` | string | Yes | Must be "start" |
+| `sampleRate` | number | No | Output sample rate (default: 24000) |
+| `encoding` | string | No | Audio encoding: "pcm16", "pcm32f" (default: "pcm16") |
+| `voice` | string | No | Voice ID or "makima" (default: "makima") |
+| `language` | string | No | Output language (default: "English") |
+| `authToken` | string | No | JWT for authenticated sessions |
+| `contractId` | string | No | Contract ID for context |
+
+**Speak Message** - Request speech synthesis
+```json
+{
+  "type": "speak",
+  "text": "Hello, I am Makima.",
+  "priority": "normal"
+}
+```
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `type` | string | Yes | Must be "speak" |
+| `text` | string | Yes | Text to synthesize |
+| `priority` | string | No | "high" or "normal" (default: "normal") |
+
+**Stop Message** - End session
+```json
+{
+  "type": "stop",
+  "reason": "user_requested"
+}
+```
+
+**Cancel Message** - Cancel current synthesis
+```json
+{
+  "type": "cancel"
+}
+```
+
+##### Server-to-Client Messages
+
+**Ready Message** - Session established
+```json
+{
+  "type": "ready",
+  "sessionId": "uuid-string",
+  "voiceLoaded": "makima",
+  "capabilities": {
+    "streaming": true,
+    "languages": ["English", "Japanese", "Chinese", "Korean", "German", "French", "Russian", "Portuguese", "Spanish", "Italian"]
+  }
+}
+```
+
+**Started Message** - TTS session configured
+```json
+{
+  "type": "started",
+  "sampleRate": 24000,
+  "encoding": "pcm16",
+  "voice": "makima"
+}
+```
+
+**AudioChunk Message** - Streaming audio data
+```json
+{
+  "type": "audioChunk",
+  "data": "<base64-encoded-audio>",
+  "sequenceNumber": 1,
+  "isFinal": false,
+  "timestampMs": 1234567890
+}
+```
+
+For binary transport (recommended for performance):
+- Server MAY send raw binary WebSocket frames
+- Binary frames contain PCM audio data directly
+- JSON control messages indicate start/end of audio stream
+
+**Complete Message** - Synthesis finished
+```json
+{
+  "type": "complete",
+  "durationMs": 1500,
+  "charactersProcessed": 25,
+  "audioLengthMs": 2100
+}
+```
+
+**Error Message** - Error occurred
+```json
+{
+  "type": "error",
+  "code": "SYNTHESIS_ERROR",
+  "message": "Failed to generate audio",
+  "recoverable": true
+}
+```
+
+**Stopped Message** - Session ended
+```json
+{
+  "type": "stopped",
+  "reason": "user_requested"
+}
+```
+
+#### 2.1.4 Integration with Listen Page
+
+The TTS endpoint SHALL integrate with the existing Listen page (`/api/v1/listen`) to enable bidirectional speech:
+
+**Bidirectional Flow:**
+```
+User Speech -> /api/v1/listen (STT) -> Transcription
+                                          |
+                                          v
+                              LLM Processing / Task Creation
+                                          |
+                                          v
+Response Text -> /api/v1/speak (TTS) -> Audio -> User
+```
+
+**Implementation Requirements:**
+1. Both endpoints SHALL support the same `contractId` for context sharing
+2. TTS SHALL support interruption when new STT input is detected
+3. Session management SHALL coordinate between STT and TTS
+
+### 2.2 Voice Configuration API
+
+#### 2.2.1 List Available Voices
+
+```
+GET /api/v1/voices
+```
+
+Response:
+```json
+{
+  "voices": [
+    {
+      "id": "makima",
+      "name": "Makima (Default)",
+      "language": "Japanese",
+      "description": "Default Makima voice (Tomori Kusunoki)",
+      "isDefault": true
+    }
+  ]
+}
+```
+
+#### 2.2.2 Upload Custom Voice (Future)
+
+```
+POST /api/v1/voices
+Content-Type: multipart/form-data
+
+audio: <audio-file>
+transcript: "Text spoken in the audio"
+name: "Custom Voice"
+```
+
+---
+
+## 3. Non-Functional Requirements
+
+### 3.1 Latency Requirements
+
+| Metric | Target | Maximum | Notes |
+|--------|--------|---------|-------|
+| First Audio Byte | < 200ms | 500ms | From text submission to first audio chunk |
+| Subsequent Chunks | < 50ms | 100ms | Inter-chunk latency |
+| End-to-End Latency | < 300ms | 800ms | Total time for short phrases |
+| Voice Prompt Loading | < 500ms | 2000ms | One-time at session start |
+
+**Measurement Points:**
+- T0: Client sends "speak" message
+- T1: First audio chunk received by client
+- T2: Last audio chunk received ("complete" message)
+- First Audio Latency = T1 - T0
+- Total Latency = T2 - T0
+
+### 3.2 Audio Quality Requirements
+
+| Specification | Value |
+|---------------|-------|
+| Output Sample Rate | 24,000 Hz |
+| Bit Depth | 16-bit (PCM16) or 32-bit float |
+| Channels | Mono (1 channel) |
+| Audio Codec | Raw PCM (WebSocket), WAV (download) |
+| Voice Similarity | > 0.90 speaker similarity score |
+
+**Quality Metrics:**
+- MOS (Mean Opinion Score): Target > 4.0
+- Speaker similarity to reference: Target > 0.90
+- No audible artifacts or glitches in streaming mode
+
+### 3.3 Hardware Requirements
+
+The TTS model runs directly in the makima process using candle.
+
+| Component | Minimum | Recommended |
+|-----------|---------|-------------|
+| GPU | CUDA-capable, 4GB VRAM (or Metal on macOS) | RTX 3060+ with 8GB+ VRAM |
+| GPU Memory | 4GB | 8GB |
+| System RAM | 8GB | 16GB |
+| Storage | 5GB (model weights) | 10GB |
+
+**GPU Memory Breakdown:**
+- Model weights (bf16): ~1.2GB
+- Speech tokenizer: ~682MB
+- KV cache during inference: ~1-2GB
+- Safety margin: ~1GB
+
+**CPU Fallback:**
+- candle supports CPU with MKL for systems without GPU
+- Latency will be higher but functional
+
+### 3.4 Scalability Requirements
+
+| Metric | Target |
+|--------|--------|
+| Concurrent Sessions | 10 per GPU |
+| Requests per Second | 50 text-to-speech requests |
+| Audio Throughput | 10 hours of audio per hour |
+
+### 3.5 Availability Requirements
+
+| Metric | Target |
+|--------|--------|
+| Service Uptime | 99.5% |
+| Recovery Time | < 30 seconds |
+| Graceful Degradation | Fall back to batch mode if streaming fails |
+
+---
+
+## 4. Architecture Specification
+
+### 4.1 System Architecture
+
+```
++-------------------------------------------------------------------------+
+|                         Client Application                               |
+|  +-------------+    +-------------+    +------------------------------+ |
+|  |   Listen    |    |   Speak     |    |        UI Components         | |
+|  |  (STT UI)   |    |  (TTS UI)   |    |  (Audio Player, Controls)    | |
+|  +------+------+    +------+------+    +------------------------------+ |
++---------|--------------------|------------------------------------------+
+          | WebSocket          | WebSocket
+          | /api/v1/listen     | /api/v1/speak
+          |                    |
++---------|--------------------|------------------------------------------+
+|         |    Makima Server (Rust/Axum)                                   |
+|  +------v--------------------v------+                                    |
+|  |        WebSocket Router          |                                    |
+|  |   (axum WebSocket handlers)      |                                    |
+|  +------+--------------------+------+                                    |
+|         |                    |                                           |
+|  +------v------+      +------v------+    +-----------------------------+ |
+|  |   Listen    |      |   Speak     |    |     Shared State            | |
+|  |   Handler   |      |   Handler   |    |  - ML Models (STT)          | |
+|  |  (STT/ML)   |      |  (TTS/ML)   |    |  - TTS Model (candle)      | |
+|  +-------------+      +------+------+    |  - Voice Prompt Cache       | |
+|                              |           |  - Session Manager          | |
+|                              |           +-----------------------------+ |
+|                       +------v------+                                    |
+|                       |  TTS Module |                                    |
+|                       |  (candle)   |                                    |
+|                       +------+------+                                    |
+|                              |                                           |
+|                       +------v------+                                    |
+|                       | Qwen3-TTS   |                                    |
+|                       | Components  |                                    |
+|                       | - LM (28L)  |                                    |
+|                       | - Code Pred |                                    |
+|                       | - Speech Tok|                                    |
+|                       +-------------+                                    |
++--------------------------------------------------------------------------+
+```
+
+### 4.2 TTS Module Structure
+
+```
+makima/src/tts/
+├── mod.rs                  // TTS trait + factory (select Chatterbox vs Qwen3)
+├── chatterbox.rs           // Existing ONNX-based Chatterbox (moved from tts.rs)
+├── qwen3/
+│   ├── mod.rs              // Qwen3TTS public API
+│   ├── model.rs            // Qwen3 LM transformer (28 layers)
+│   ├── code_predictor.rs   // MTP module (5 layers, 16 codebooks)
+│   ├── speech_tokenizer.rs // Encoder + Decoder (causal ConvNet)
+│   ├── config.rs           // Model config from config.json
+│   └── generate.rs         // Autoregressive generation loop with KV cache
+```
+
+#### 4.2.1 TTS Trait
+
+```rust
+// makima/src/tts/mod.rs
+
+/// Trait for text-to-speech implementations.
+#[async_trait]
+pub trait TtsEngine: Send + Sync {
+    /// Generate audio from text with a given voice prompt.
+    async fn generate(
+        &self,
+        text: &str,
+        voice_id: &str,
+        language: &str,
+    ) -> Result<Vec<AudioChunk>, TtsError>;
+
+    /// Load and cache a voice prompt from reference audio.
+    async fn load_voice(&self, voice_id: &str) -> Result<(), TtsError>;
+
+    /// Check if the engine is ready for inference.
+    fn is_ready(&self) -> bool;
+}
+
+/// Select the appropriate TTS engine based on configuration.
+pub fn create_engine(config: &TtsConfig) -> Box<dyn TtsEngine> {
+    match config.engine {
+        TtsEngineType::Qwen3 => Box::new(qwen3::Qwen3Tts::new(config)),
+        TtsEngineType::Chatterbox => Box::new(chatterbox::ChatterboxTts::new(config)),
+    }
+}
+```
+
+#### 4.2.2 Qwen3 Candle Implementation
+
+The Qwen3 module implements the three core model components using candle:
+
+1. **Language Model** — 28-layer transformer using candle-transformers' Qwen2 attention with TTS-specific modifications
+2. **Code Predictor** — 5-layer MTP module predicting 16 codebook layers
+3. **Speech Tokenizer** — GAN-based codec with Conv1d encoder/decoder
+
+**Key candle features used:**
+- `candle_core::Tensor` for all tensor operations
+- `candle_nn::Module` for model layers
+- `candle_nn::VarBuilder` for loading safetensors weights
+- `candle_core::Device` for GPU/CPU selection
+
+#### 4.2.3 Model Loading
+
+Models are loaded lazily on first TTS request, following the pattern established by `listen.rs`:
+
+```rust
+// Models held in SharedState behind async mutex
+pub struct TtsModels {
+    pub engine: Box<dyn TtsEngine>,
+    pub voice_cache: VoicePromptCache,
+}
+
+impl AppState {
+    pub async fn get_tts_models(&self) -> Result<&TtsModels, TtsError> {
+        self.tts_models.get_or_try_init(|| async {
+            // Load safetensors weights via candle
+            // Initialize voice cache with default Makima voice
+        }).await
+    }
+}
+```
+
+### 4.3 Speak Handler
+
+```rust
+// makima/src/server/handlers/speak.rs
+
+/// WebSocket handler for TTS streaming.
+/// Calls the TTS engine directly — no proxy, no external service.
+pub async fn websocket_handler(
+    ws: WebSocketUpgrade,
+    State(state): State<SharedState>,
+) -> Response {
+    ws.on_upgrade(|socket| handle_speak_socket(socket, state))
+}
+
+async fn handle_speak_socket(socket: WebSocket, state: SharedState) {
+    let session_id = Uuid::new_v4().to_string();
+
+    // Get or lazily load TTS models
+    let tts = match state.get_tts_models().await {
+        Ok(tts) => tts,
+        Err(e) => {
+            // Send error and close
+            return;
+        }
+    };
+
+    // Handle WebSocket messages directly
+    // Parse JSON commands, run inference, stream audio chunks back
+}
+```
+
+### 4.4 Voice Prompt Caching
+
+Voice prompts are cached in-memory using an LRU cache:
+
+```rust
+// makima/src/tts/mod.rs
+
+pub struct VoicePromptCache {
+    cache: tokio::sync::Mutex<lru::LruCache<String, VoicePrompt>>,
+}
+
+impl VoicePromptCache {
+    pub fn new(max_size: usize) -> Self { /* ... */ }
+    pub async fn get(&self, voice_id: &str) -> Option<VoicePrompt> { /* ... */ }
+    pub async fn insert(&self, voice_id: String, prompt: VoicePrompt) { /* ... */ }
+}
+```
+
+### 4.5 Error Handling and Recovery
+
+#### 4.5.1 Error Categories
+
+| Error Code | Category | Recoverable | Action |
+|------------|----------|-------------|--------|
+| `MODEL_LOADING` | Initialization | Yes | Wait and retry |
+| `SYNTHESIS_ERROR` | Generation | Yes | Retry with same input |
+| `INVALID_TEXT` | Input | No | Return error to client |
+| `VOICE_NOT_FOUND` | Configuration | No | Fall back to default voice |
+| `GPU_OUT_OF_MEMORY` | Resource | Yes | Clear cache, retry on CPU |
+| `TIMEOUT` | Inference | Yes | Retry with backoff |
+
+---
+
+## 5. API Contract
+
+### 5.1 WebSocket Message Formats
+
+#### 5.1.1 Client-to-Server Messages
+
+```typescript
+// TypeScript type definitions for client implementation
+
+interface StartMessage {
+  type: "start";
+  sampleRate?: number;      // Default: 24000
+  encoding?: "pcm16" | "pcm32f";  // Default: "pcm16"
+  voice?: string;           // Default: "makima"
+  language?: string;        // Default: "English"
+  authToken?: string;       // JWT for authenticated sessions
+  contractId?: string;      // Contract context
+}
+
+interface SpeakMessage {
+  type: "speak";
+  text: string;             // Required: text to synthesize
+  priority?: "normal" | "high";  // Default: "normal"
+}
+
+interface CancelMessage {
+  type: "cancel";
+}
+
+interface StopMessage {
+  type: "stop";
+  reason?: string;
+}
+
+type ClientMessage = StartMessage | SpeakMessage | CancelMessage | StopMessage;
+```
+
+#### 5.1.2 Server-to-Client Messages
+
+```typescript
+interface ReadyMessage {
+  type: "ready";
+  sessionId: string;
+  voiceLoaded: string;
+  capabilities: {
+    streaming: boolean;
+    languages: string[];
+  };
+}
+
+interface StartedMessage {
+  type: "started";
+  sampleRate: number;
+  encoding: string;
+  voice: string;
+}
+
+interface AudioChunkMessage {
+  type: "audioChunk";
+  data: string;           // Base64-encoded PCM audio
+  sequenceNumber: number;
+  isFinal: boolean;
+  timestampMs: number;
+}
+
+interface CompleteMessage {
+  type: "complete";
+  durationMs: number;
+  charactersProcessed: number;
+  audioLengthMs: number;
+}
+
+interface ErrorMessage {
+  type: "error";
+  code: string;
+  message: string;
+  recoverable: boolean;
+}
+
+interface StoppedMessage {
+  type: "stopped";
+  reason: string;
+}
+
+type ServerMessage =
+  | ReadyMessage
+  | StartedMessage
+  | AudioChunkMessage
+  | CompleteMessage
+  | ErrorMessage
+  | StoppedMessage;
+```
+
+### 5.2 Error Codes
+
+| Code | HTTP-like | Description | Recovery |
+|------|-----------|-------------|----------|
+| `MODEL_LOADING` | 503 | Model still loading | Wait and retry |
+| `SYNTHESIS_ERROR` | 500 | Failed to generate audio | Retry |
+| `INVALID_TEXT` | 400 | Text is empty or invalid | Fix input |
+| `VOICE_NOT_FOUND` | 404 | Requested voice doesn't exist | Use default |
+| `UNAUTHORIZED` | 401 | Invalid or missing auth token | Re-authenticate |
+| `RATE_LIMITED` | 429 | Too many requests | Back off |
+| `TIMEOUT` | 408 | Operation timed out | Retry |
+| `CANCELLED` | 499 | Client cancelled request | N/A |
+
+### 5.3 Session Management
+
+#### 5.3.1 Session Lifecycle
+
+```
+DISCONNECTED -> CONNECTING -> READY -> STARTED -> SPEAKING -> READY -> ... -> STOPPED
+                    |            |         |                      |
+                    v            v         v                      v
+                  ERROR       ERROR     ERROR                  STOPPED
+```
+
+---
+
+## 6. Voice Asset Requirements
+
+### 6.1 Makima Voice Clip Specifications
+
+#### 6.1.1 Audio Requirements
+
+| Specification | Requirement |
+|---------------|-------------|
+| Duration | 5-10 seconds (minimum 3s) |
+| Format | WAV (PCM) |
+| Sample Rate | 24,000 Hz or higher |
+| Bit Depth | 16-bit or higher |
+| Channels | Mono (preferred) or Stereo |
+| Content | Clear speech, natural tone |
+| Background | Minimal noise/music |
+
+#### 6.1.2 Content Guidelines
+
+**DO:**
+- Use dialogue with varied intonation
+- Include multiple phonemes
+- Capture natural speaking rhythm
+- Extract from clean audio scenes
+
+**DON'T:**
+- Include background music
+- Use shouting or whispering
+- Include sound effects
+- Use heavily processed audio
+
+#### 6.1.3 Transcript Requirements
+
+| Specification | Requirement |
+|---------------|-------------|
+| Format | Plain text (.txt) or JSON |
+| Encoding | UTF-8 |
+| Content | Exact transcription of audio |
+| Language | Japanese (for Japanese reference) |
+
+### 6.2 Storage Location and Management
+
+#### 6.2.1 Directory Structure
+
+```
+models/
+└── voices/
+    ├── makima/
+    │   ├── reference.wav       # Primary reference audio
+    │   ├── transcript.txt      # Plain text transcript
+    │   ├── transcript.json     # Structured transcript (optional)
+    │   └── metadata.json       # Voice metadata
+    ├── makima-alt/             # Alternative Makima clips (future)
+    │   └── ...
+    └── custom/                 # User-uploaded voices (future)
+        └── {voice_id}/
+            ├── reference.wav
+            ├── transcript.txt
+            └── metadata.json
+```
+
+---
+
+## 7. Testing Strategy
+
+### 7.1 Unit Tests
+
+#### 7.1.1 Rust TTS Module Tests
+
+```rust
+// makima/src/tts/qwen3/tests.rs
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_config_loading() {
+        let config = Qwen3Config::from_json("test_config.json").unwrap();
+        assert_eq!(config.hidden_size, 1024);
+        assert_eq!(config.num_layers, 28);
+    }
+
+    #[test]
+    fn test_voice_prompt_cache_lru() {
+        let cache = VoicePromptCache::new(2);
+        cache.insert("a", prompt_a);
+        cache.insert("b", prompt_b);
+        cache.get("a"); // access a
+        cache.insert("c", prompt_c); // should evict b
+
+        assert!(cache.get("a").is_some());
+        assert!(cache.get("b").is_none());
+        assert!(cache.get("c").is_some());
+    }
+
+    #[tokio::test]
+    async fn test_speak_handler_message_parsing() {
+        let json = r#"{"type": "start", "voice": "makima"}"#;
+        let msg: SpeakClientMessage = serde_json::from_str(json).unwrap();
+
+        match msg {
+            SpeakClientMessage::Start(start) => {
+                assert_eq!(start.voice, Some("makima".to_string()));
+            }
+            _ => panic!("Expected Start message"),
+        }
+    }
+}
+```
+
+### 7.2 Integration Tests
+
+```rust
+// tests/tts_integration.rs
+
+#[tokio::test]
+async fn test_speak_websocket_flow() {
+    // Start test server with TTS enabled
+    let state = create_test_state_with_tts().await;
+    let app = make_router(state);
+
+    // Connect WebSocket
+    let ws = connect_ws("/api/v1/speak").await;
+
+    // Send start
+    ws.send_json(json!({"type": "start", "voice": "makima"})).await;
+    let ready = ws.recv_json().await;
+    assert_eq!(ready["type"], "ready");
+
+    // Send speak
+    ws.send_json(json!({"type": "speak", "text": "Hello."})).await;
+
+    // Collect audio chunks
+    let mut chunks = vec![];
+    loop {
+        let msg = ws.recv().await;
+        match msg {
+            WsMsg::Binary(data) => chunks.push(data),
+            WsMsg::Text(json) => {
+                let data: Value = serde_json::from_str(&json).unwrap();
+                if data["type"] == "complete" { break; }
+            }
+        }
+    }
+    assert!(!chunks.is_empty());
+}
+```
+
+### 7.3 Performance Targets
+
+| Metric | Target | Acceptable | Warning |
+|--------|--------|------------|---------|
+| First Audio (short) | < 150ms | < 200ms | > 300ms |
+| First Audio (medium) | < 200ms | < 300ms | > 500ms |
+| First Audio (long) | < 300ms | < 500ms | > 800ms |
+| Inter-chunk | < 30ms | < 50ms | > 100ms |
+| Memory (GPU) | < 4GB | < 6GB | > 8GB |
+| Memory (CPU) | < 2GB | < 4GB | > 8GB |
+
+---
+
+## 8. Implementation Phases
+
+### Phase 1: Candle-Based Qwen3-TTS Module (Week 1-2)
+
+**Deliverables:**
+- [ ] `makima/src/tts/mod.rs` — TTS trait + factory
+- [ ] `makima/src/tts/chatterbox.rs` — Move existing code from tts.rs
+- [ ] `makima/src/tts/qwen3/model.rs` — 28-layer LM backbone (extend candle Qwen2)
+- [ ] `makima/src/tts/qwen3/code_predictor.rs` — MTP module (5 layers, 16 codebooks)
+- [ ] `makima/src/tts/qwen3/speech_tokenizer.rs` — ConvNet encoder/decoder + RVQ
+- [ ] `makima/src/tts/qwen3/config.rs` — Config from safetensors
+- [ ] `makima/src/tts/qwen3/generate.rs` — Autoregressive generation with KV cache
+- [ ] Add `candle-core`, `candle-nn`, `candle-transformers` to Cargo.toml
+
+**Success Criteria:**
+- Model loads safetensors weights successfully
+- Can generate audio from text via direct inference
+- First audio latency < 500ms (initial, unoptimized)
+
+### Phase 2: WebSocket Handler + Voice Assets (Week 2-3)
+
+**Deliverables:**
+- [ ] Update `makima/src/server/handlers/speak.rs` — Direct TTS handler (no proxy)
+- [ ] Lazy model loading via `SharedState`
+- [ ] Voice prompt caching
+- [ ] Makima voice asset acquisition and processing
+- [ ] Basic error handling and session management
+
+**Success Criteria:**
+- `/api/v1/speak` endpoint produces streaming audio
+- Default Makima voice works
+- Error handling matches specification
+
+### Phase 3: Optimization + Integration (Week 3-4)
+
+**Deliverables:**
+- [ ] Streaming audio generation (token-by-token decoding)
+- [ ] GPU memory optimization
+- [ ] Listen page integration for bidirectional speech
+- [ ] Session coordination between STT and TTS
+- [ ] Full test suite (unit, integration)
+- [ ] Latency benchmarks
+
+**Success Criteria:**
+- First audio latency < 200ms
+- Memory usage < 6GB
+- All tests passing
+- Documentation complete
+
+---
+
+## Appendix
+
+### A. Dependencies
+
+#### Rust (Cargo.toml additions)
+
+```toml
+[dependencies]
+candle-core = "0.8"
+candle-nn = "0.8"
+candle-transformers = "0.8"
+# Keep existing: tokenizers, hf-hub, ndarray (for compatibility)
+```
+
+### B. Environment Variables
+
+```bash
+# TTS Configuration
+TTS_ENGINE=qwen3                    # "qwen3" or "chatterbox"
+TTS_MODEL_ID=Qwen/Qwen3-TTS-12Hz-0.6B-Base
+TTS_DEVICE=cuda:0                   # "cuda:0", "metal", or "cpu"
+TTS_VOICES_DIR=models/voices
+TTS_DEFAULT_VOICE=makima
+```
+
+### C. References
+
+1. [Qwen3-TTS-12Hz-0.6B-Base (Hugging Face)](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-0.6B-Base)
+2. [Qwen3-TTS GitHub Repository](https://github.com/QwenLM/Qwen3-TTS)
+3. [Qwen3-TTS Technical Report (arXiv:2601.15621)](https://arxiv.org/abs/2601.15621)
+4. [Candle — HuggingFace Rust ML Framework](https://github.com/huggingface/candle)
+5. [axum WebSocket Documentation](https://docs.rs/axum/latest/axum/extract/ws/index.html)
+6. [docs/research/rust-native-tts-research.md](../research/rust-native-tts-research.md) — Detailed feasibility analysis
+
+### D. Glossary
+
+| Term | Definition |
+|------|------------|
+| **TTS** | Text-to-Speech: Converting text input to audio output |
+| **STT** | Speech-to-Text: Converting audio input to text output |
+| **Voice Cloning** | Creating synthetic speech that mimics a specific speaker |
+| **Voice Prompt** | Pre-computed speaker embedding for voice cloning |
+| **Candle** | HuggingFace's minimalist Rust ML framework |
+| **SafeTensors** | Efficient, safe model weight serialization format |
+| **RVQ** | Residual Vector Quantization — multi-codebook audio tokenization |
+| **MTP** | Multi-Token Prediction — code predictor generating 16 codebook layers |
+| **bf16** | Brain floating-point 16-bit format for GPU computation |