# Qwen3-TTS Integration Specification
**Version:** 2.0
**Date:** 2026-01-27
**Status:** Draft
**Author:** Makima Engineering
## Table of Contents
1. [Overview](#1-overview)
2. [Functional Requirements](#2-functional-requirements)
3. [Non-Functional Requirements](#3-non-functional-requirements)
4. [Architecture Specification](#4-architecture-specification)
5. [API Contract](#5-api-contract)
6. [Voice Asset Requirements](#6-voice-asset-requirements)
7. [Testing Strategy](#7-testing-strategy)
8. [Implementation Phases](#8-implementation-phases)
9. [Appendix](#appendix)
---
## 1. Overview
### 1.1 Purpose
This specification defines the integration of Qwen3-TTS-12Hz-0.6B-Base as a replacement for the existing Chatterbox-Turbo TTS implementation in the makima system. The new implementation is a **pure Rust** solution using the **candle** ML framework — no Python, no separate microservice. The TTS model runs directly inside the main makima process. The implementation will provide:
- **Streaming TTS** with near-real-time audio synthesis
- **Voice cloning** with default Makima voice (Japanese voice actress speaking English)
- **Bidirectional speech integration** for the Listen page
- **WebSocket-based streaming API** for low-latency delivery
### 1.2 Background
The current TTS implementation (Chatterbox-Turbo-ONNX) has limitations:
- No streaming support (batch-only generation)
- No HTTP/WebSocket endpoint exposed
- High latency for interactive use cases
Qwen3-TTS offers significant improvements:
- **97ms** end-to-end latency (vs. batch processing)
- **10 languages** supported including Japanese cross-lingual cloning
- **3-second** reference audio for voice cloning
- **Dual-track streaming architecture**
### 1.3 Scope
This specification covers:
- WebSocket endpoint `/api/v1/speak` for streaming TTS
- Pure Rust candle-based model inference running in-process
- Voice asset management
- Testing and benchmarking
Out of scope:
- ONNX export of Qwen3-TTS (not available)
- Instruction-following TTS features (base model only)
- Full replacement of STT/Listen functionality
---
## 2. Functional Requirements
### 2.1 WebSocket Endpoint: `/api/v1/speak`
The TTS service SHALL be exposed via a WebSocket endpoint at `/api/v1/speak` for streaming audio synthesis.
#### 2.1.1 Connection Flow
```
Client Server (Rust/Axum)
| |
|--- WS Connect ------------------>|
| | [Load TTS model lazily if needed]
|<-- Ready (session_id) -----------|
| |
|--- Start (config) -------------->|
|<-- Started ----------------------| [Load voice prompt]
| |
|--- Speak (text) ---------------->| [Direct candle model inference]
| |
|<-- AudioChunk (binary) ----------|
|<-- AudioChunk (binary) ----------|
|<-- Complete ---------------------|
| |
|--- Stop ------------------------>|
|<-- Stopped ----------------------|
| |
```
#### 2.1.2 Voice Cloning
The system SHALL support voice cloning with the following modes:
| Mode | Description | Requirements |
|------|-------------|--------------|
| Default (Makima) | Pre-loaded Makima voice | None (auto-selected) |
| Custom Voice | User-provided reference | Audio file + transcript |
| X-Vector Only | Speaker embedding only | Audio file (no transcript) |
**Default Voice Behavior:**
- If no voice is specified, use the pre-loaded Makima voice prompt
- Makima voice SHALL be a Japanese voice actress (Tomori Kusunoki) speaking English
- Voice prompt is pre-computed at model load time for zero-latency switching
#### 2.1.3 Message Protocol
##### Client-to-Server Messages
All messages use JSON format with a `type` field for routing.
**Start Message** - Initialize TTS session
```json
{
"type": "start",
"sampleRate": 24000,
"encoding": "pcm16",
"voice": "makima",
"language": "English",
"authToken": "optional-jwt-token",
"contractId": "optional-contract-uuid"
}
```
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `type` | string | Yes | Must be "start" |
| `sampleRate` | number | No | Output sample rate (default: 24000) |
| `encoding` | string | No | Audio encoding: "pcm16", "pcm32f" (default: "pcm16") |
| `voice` | string | No | Voice ID or "makima" (default: "makima") |
| `language` | string | No | Output language (default: "English") |
| `authToken` | string | No | JWT for authenticated sessions |
| `contractId` | string | No | Contract ID for context |
**Speak Message** - Request speech synthesis
```json
{
"type": "speak",
"text": "Hello, I am Makima.",
"priority": "normal"
}
```
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `type` | string | Yes | Must be "speak" |
| `text` | string | Yes | Text to synthesize |
| `priority` | string | No | "high" or "normal" (default: "normal") |
**Stop Message** - End session
```json
{
"type": "stop",
"reason": "user_requested"
}
```
**Cancel Message** - Cancel current synthesis
```json
{
"type": "cancel"
}
```
##### Server-to-Client Messages
**Ready Message** - Session established
```json
{
"type": "ready",
"sessionId": "uuid-string",
"voiceLoaded": "makima",
"capabilities": {
"streaming": true,
"languages": ["English", "Japanese", "Chinese", "Korean", "German", "French", "Russian", "Portuguese", "Spanish", "Italian"]
}
}
```
**Started Message** - TTS session configured
```json
{
"type": "started",
"sampleRate": 24000,
"encoding": "pcm16",
"voice": "makima"
}
```
**AudioChunk Message** - Streaming audio data
```json
{
"type": "audioChunk",
"data": "<base64-encoded-audio>",
"sequenceNumber": 1,
"isFinal": false,
"timestampMs": 1234567890
}
```
For binary transport (recommended for performance):
- Server MAY send raw binary WebSocket frames
- Binary frames contain PCM audio data directly
- JSON control messages indicate start/end of audio stream
**Complete Message** - Synthesis finished
```json
{
"type": "complete",
"durationMs": 1500,
"charactersProcessed": 25,
"audioLengthMs": 2100
}
```
**Error Message** - Error occurred
```json
{
"type": "error",
"code": "SYNTHESIS_ERROR",
"message": "Failed to generate audio",
"recoverable": true
}
```
**Stopped Message** - Session ended
```json
{
"type": "stopped",
"reason": "user_requested"
}
```
#### 2.1.4 Integration with Listen Page
The TTS endpoint SHALL integrate with the existing Listen page (`/api/v1/listen`) to enable bidirectional speech:
**Bidirectional Flow:**
```
User Speech -> /api/v1/listen (STT) -> Transcription
|
v
LLM Processing / Task Creation
|
v
Response Text -> /api/v1/speak (TTS) -> Audio -> User
```
**Implementation Requirements:**
1. Both endpoints SHALL support the same `contractId` for context sharing
2. TTS SHALL support interruption when new STT input is detected
3. Session management SHALL coordinate between STT and TTS
### 2.2 Voice Configuration API
#### 2.2.1 List Available Voices
```
GET /api/v1/voices
```
Response:
```json
{
"voices": [
{
"id": "makima",
"name": "Makima (Default)",
"language": "Japanese",
"description": "Default Makima voice (Tomori Kusunoki)",
"isDefault": true
}
]
}
```
#### 2.2.2 Upload Custom Voice (Future)
```
POST /api/v1/voices
Content-Type: multipart/form-data
audio: <audio-file>
transcript: "Text spoken in the audio"
name: "Custom Voice"
```
---
## 3. Non-Functional Requirements
### 3.1 Latency Requirements
| Metric | Target | Maximum | Notes |
|--------|--------|---------|-------|
| First Audio Byte | < 200ms | 500ms | From text submission to first audio chunk |
| Subsequent Chunks | < 50ms | 100ms | Inter-chunk latency |
| End-to-End Latency | < 300ms | 800ms | Total time for short phrases |
| Voice Prompt Loading | < 500ms | 2000ms | One-time at session start |
**Measurement Points:**
- T0: Client sends "speak" message
- T1: First audio chunk received by client
- T2: Last audio chunk received ("complete" message)
- First Audio Latency = T1 - T0
- Total Latency = T2 - T0
### 3.2 Audio Quality Requirements
| Specification | Value |
|---------------|-------|
| Output Sample Rate | 24,000 Hz |
| Bit Depth | 16-bit (PCM16) or 32-bit float |
| Channels | Mono (1 channel) |
| Audio Codec | Raw PCM (WebSocket), WAV (download) |
| Voice Similarity | > 0.90 speaker similarity score |
**Quality Metrics:**
- MOS (Mean Opinion Score): Target > 4.0
- Speaker similarity to reference: Target > 0.90
- No audible artifacts or glitches in streaming mode
### 3.3 Hardware Requirements
The TTS model runs directly in the makima process using candle.
| Component | Minimum | Recommended |
|-----------|---------|-------------|
| GPU | CUDA-capable, 4GB VRAM (or Metal on macOS) | RTX 3060+ with 8GB+ VRAM |
| GPU Memory | 4GB | 8GB |
| System RAM | 8GB | 16GB |
| Storage | 5GB (model weights) | 10GB |
**GPU Memory Breakdown:**
- Model weights (bf16): ~1.2GB
- Speech tokenizer: ~682MB
- KV cache during inference: ~1-2GB
- Safety margin: ~1GB
**CPU Fallback:**
- candle supports CPU with MKL for systems without GPU
- Latency will be higher but functional
### 3.4 Scalability Requirements
| Metric | Target |
|--------|--------|
| Concurrent Sessions | 10 per GPU |
| Requests per Second | 50 text-to-speech requests |
| Audio Throughput | 10 hours of audio per hour |
### 3.5 Availability Requirements
| Metric | Target |
|--------|--------|
| Service Uptime | 99.5% |
| Recovery Time | < 30 seconds |
| Graceful Degradation | Fall back to batch mode if streaming fails |
---
## 4. Architecture Specification
### 4.1 System Architecture
```
+-------------------------------------------------------------------------+
| Client Application |
| +-------------+ +-------------+ +------------------------------+ |
| | Listen | | Speak | | UI Components | |
| | (STT UI) | | (TTS UI) | | (Audio Player, Controls) | |
| +------+------+ +------+------+ +------------------------------+ |
+---------|--------------------|------------------------------------------+
| WebSocket | WebSocket
| /api/v1/listen | /api/v1/speak
| |
+---------|--------------------|------------------------------------------+
| | Makima Server (Rust/Axum) |
| +------v--------------------v------+ |
| | WebSocket Router | |
| | (axum WebSocket handlers) | |
| +------+--------------------+------+ |
| | | |
| +------v------+ +------v------+ +-----------------------------+ |
| | Listen | | Speak | | Shared State | |
| | Handler | | Handler | | - ML Models (STT) | |
| | (STT/ML) | | (TTS/ML) | | - TTS Model (candle) | |
| +-------------+ +------+------+ | - Voice Prompt Cache | |
| | | - Session Manager | |
| | +-----------------------------+ |
| +------v------+ |
| | TTS Module | |
| | (candle) | |
| +------+------+ |
| | |
| +------v------+ |
| | Qwen3-TTS | |
| | Components | |
| | - LM (28L) | |
| | - Code Pred | |
| | - Speech Tok| |
| +-------------+ |
+--------------------------------------------------------------------------+
```
### 4.2 TTS Module Structure
```
makima/src/tts/
├── mod.rs // TTS trait + factory (select Chatterbox vs Qwen3)
├── chatterbox.rs // Existing ONNX-based Chatterbox (moved from tts.rs)
├── qwen3/
│ ├── mod.rs // Qwen3TTS public API
│ ├── model.rs // Qwen3 LM transformer (28 layers)
│ ├── code_predictor.rs // MTP module (5 layers, 16 codebooks)
│ ├── speech_tokenizer.rs // Encoder + Decoder (causal ConvNet)
│ ├── config.rs // Model config from config.json
│ └── generate.rs // Autoregressive generation loop with KV cache
```
#### 4.2.1 TTS Trait
```rust
// makima/src/tts/mod.rs
/// Trait for text-to-speech implementations.
#[async_trait]
pub trait TtsEngine: Send + Sync {
/// Generate audio from text with a given voice prompt.
async fn generate(
&self,
text: &str,
voice_id: &str,
language: &str,
) -> Result<Vec<AudioChunk>, TtsError>;
/// Load and cache a voice prompt from reference audio.
async fn load_voice(&self, voice_id: &str) -> Result<(), TtsError>;
/// Check if the engine is ready for inference.
fn is_ready(&self) -> bool;
}
/// Select the appropriate TTS engine based on configuration.
pub fn create_engine(config: &TtsConfig) -> Box<dyn TtsEngine> {
match config.engine {
TtsEngineType::Qwen3 => Box::new(qwen3::Qwen3Tts::new(config)),
TtsEngineType::Chatterbox => Box::new(chatterbox::ChatterboxTts::new(config)),
}
}
```
#### 4.2.2 Qwen3 Candle Implementation
The Qwen3 module implements the three core model components using candle:
1. **Language Model** — 28-layer transformer using candle-transformers' Qwen2 attention with TTS-specific modifications
2. **Code Predictor** — 5-layer MTP module predicting 16 codebook layers
3. **Speech Tokenizer** — GAN-based codec with Conv1d encoder/decoder
**Key candle features used:**
- `candle_core::Tensor` for all tensor operations
- `candle_nn::Module` for model layers
- `candle_nn::VarBuilder` for loading safetensors weights
- `candle_core::Device` for GPU/CPU selection
#### 4.2.3 Model Loading
Models are loaded lazily on first TTS request, following the pattern established by `listen.rs`:
```rust
// Models held in SharedState behind async mutex
pub struct TtsModels {
pub engine: Box<dyn TtsEngine>,
pub voice_cache: VoicePromptCache,
}
impl AppState {
pub async fn get_tts_models(&self) -> Result<&TtsModels, TtsError> {
self.tts_models.get_or_try_init(|| async {
// Load safetensors weights via candle
// Initialize voice cache with default Makima voice
}).await
}
}
```
### 4.3 Speak Handler
```rust
// makima/src/server/handlers/speak.rs
/// WebSocket handler for TTS streaming.
/// Calls the TTS engine directly — no proxy, no external service.
pub async fn websocket_handler(
ws: WebSocketUpgrade,
State(state): State<SharedState>,
) -> Response {
ws.on_upgrade(|socket| handle_speak_socket(socket, state))
}
async fn handle_speak_socket(socket: WebSocket, state: SharedState) {
let session_id = Uuid::new_v4().to_string();
// Get or lazily load TTS models
let tts = match state.get_tts_models().await {
Ok(tts) => tts,
Err(e) => {
// Send error and close
return;
}
};
// Handle WebSocket messages directly
// Parse JSON commands, run inference, stream audio chunks back
}
```
### 4.4 Voice Prompt Caching
Voice prompts are cached in-memory using an LRU cache:
```rust
// makima/src/tts/mod.rs
pub struct VoicePromptCache {
cache: tokio::sync::Mutex<lru::LruCache<String, VoicePrompt>>,
}
impl VoicePromptCache {
pub fn new(max_size: usize) -> Self { /* ... */ }
pub async fn get(&self, voice_id: &str) -> Option<VoicePrompt> { /* ... */ }
pub async fn insert(&self, voice_id: String, prompt: VoicePrompt) { /* ... */ }
}
```
### 4.5 Error Handling and Recovery
#### 4.5.1 Error Categories
| Error Code | Category | Recoverable | Action |
|------------|----------|-------------|--------|
| `MODEL_LOADING` | Initialization | Yes | Wait and retry |
| `SYNTHESIS_ERROR` | Generation | Yes | Retry with same input |
| `INVALID_TEXT` | Input | No | Return error to client |
| `VOICE_NOT_FOUND` | Configuration | No | Fall back to default voice |
| `GPU_OUT_OF_MEMORY` | Resource | Yes | Clear cache, retry on CPU |
| `TIMEOUT` | Inference | Yes | Retry with backoff |
---
## 5. API Contract
### 5.1 WebSocket Message Formats
#### 5.1.1 Client-to-Server Messages
```typescript
// TypeScript type definitions for client implementation
interface StartMessage {
type: "start";
sampleRate?: number; // Default: 24000
encoding?: "pcm16" | "pcm32f"; // Default: "pcm16"
voice?: string; // Default: "makima"
language?: string; // Default: "English"
authToken?: string; // JWT for authenticated sessions
contractId?: string; // Contract context
}
interface SpeakMessage {
type: "speak";
text: string; // Required: text to synthesize
priority?: "normal" | "high"; // Default: "normal"
}
interface CancelMessage {
type: "cancel";
}
interface StopMessage {
type: "stop";
reason?: string;
}
type ClientMessage = StartMessage | SpeakMessage | CancelMessage | StopMessage;
```
#### 5.1.2 Server-to-Client Messages
```typescript
interface ReadyMessage {
type: "ready";
sessionId: string;
voiceLoaded: string;
capabilities: {
streaming: boolean;
languages: string[];
};
}
interface StartedMessage {
type: "started";
sampleRate: number;
encoding: string;
voice: string;
}
interface AudioChunkMessage {
type: "audioChunk";
data: string; // Base64-encoded PCM audio
sequenceNumber: number;
isFinal: boolean;
timestampMs: number;
}
interface CompleteMessage {
type: "complete";
durationMs: number;
charactersProcessed: number;
audioLengthMs: number;
}
interface ErrorMessage {
type: "error";
code: string;
message: string;
recoverable: boolean;
}
interface StoppedMessage {
type: "stopped";
reason: string;
}
type ServerMessage =
| ReadyMessage
| StartedMessage
| AudioChunkMessage
| CompleteMessage
| ErrorMessage
| StoppedMessage;
```
### 5.2 Error Codes
| Code | HTTP-like | Description | Recovery |
|------|-----------|-------------|----------|
| `MODEL_LOADING` | 503 | Model still loading | Wait and retry |
| `SYNTHESIS_ERROR` | 500 | Failed to generate audio | Retry |
| `INVALID_TEXT` | 400 | Text is empty or invalid | Fix input |
| `VOICE_NOT_FOUND` | 404 | Requested voice doesn't exist | Use default |
| `UNAUTHORIZED` | 401 | Invalid or missing auth token | Re-authenticate |
| `RATE_LIMITED` | 429 | Too many requests | Back off |
| `TIMEOUT` | 408 | Operation timed out | Retry |
| `CANCELLED` | 499 | Client cancelled request | N/A |
### 5.3 Session Management
#### 5.3.1 Session Lifecycle
```
DISCONNECTED -> CONNECTING -> READY -> STARTED -> SPEAKING -> READY -> ... -> STOPPED
| | | |
v v v v
ERROR ERROR ERROR STOPPED
```
---
## 6. Voice Asset Requirements
### 6.1 Makima Voice Clip Specifications
#### 6.1.1 Audio Requirements
| Specification | Requirement |
|---------------|-------------|
| Duration | 5-10 seconds (minimum 3s) |
| Format | WAV (PCM) |
| Sample Rate | 24,000 Hz or higher |
| Bit Depth | 16-bit or higher |
| Channels | Mono (preferred) or Stereo |
| Content | Clear speech, natural tone |
| Background | Minimal noise/music |
#### 6.1.2 Content Guidelines
**DO:**
- Use dialogue with varied intonation
- Include multiple phonemes
- Capture natural speaking rhythm
- Extract from clean audio scenes
**DON'T:**
- Include background music
- Use shouting or whispering
- Include sound effects
- Use heavily processed audio
#### 6.1.3 Transcript Requirements
| Specification | Requirement |
|---------------|-------------|
| Format | Plain text (.txt) or JSON |
| Encoding | UTF-8 |
| Content | Exact transcription of audio |
| Language | Japanese (for Japanese reference) |
### 6.2 Storage Location and Management
#### 6.2.1 Directory Structure
```
models/
└── voices/
├── makima/
│ ├── reference.wav # Primary reference audio
│ ├── transcript.txt # Plain text transcript
│ ├── transcript.json # Structured transcript (optional)
│ └── metadata.json # Voice metadata
├── makima-alt/ # Alternative Makima clips (future)
│ └── ...
└── custom/ # User-uploaded voices (future)
└── {voice_id}/
├── reference.wav
├── transcript.txt
└── metadata.json
```
---
## 7. Testing Strategy
### 7.1 Unit Tests
#### 7.1.1 Rust TTS Module Tests
```rust
// makima/src/tts/qwen3/tests.rs
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_config_loading() {
let config = Qwen3Config::from_json("test_config.json").unwrap();
assert_eq!(config.hidden_size, 1024);
assert_eq!(config.num_layers, 28);
}
#[test]
fn test_voice_prompt_cache_lru() {
let cache = VoicePromptCache::new(2);
cache.insert("a", prompt_a);
cache.insert("b", prompt_b);
cache.get("a"); // access a
cache.insert("c", prompt_c); // should evict b
assert!(cache.get("a").is_some());
assert!(cache.get("b").is_none());
assert!(cache.get("c").is_some());
}
#[tokio::test]
async fn test_speak_handler_message_parsing() {
let json = r#"{"type": "start", "voice": "makima"}"#;
let msg: SpeakClientMessage = serde_json::from_str(json).unwrap();
match msg {
SpeakClientMessage::Start(start) => {
assert_eq!(start.voice, Some("makima".to_string()));
}
_ => panic!("Expected Start message"),
}
}
}
```
### 7.2 Integration Tests
```rust
// tests/tts_integration.rs
#[tokio::test]
async fn test_speak_websocket_flow() {
// Start test server with TTS enabled
let state = create_test_state_with_tts().await;
let app = make_router(state);
// Connect WebSocket
let ws = connect_ws("/api/v1/speak").await;
// Send start
ws.send_json(json!({"type": "start", "voice": "makima"})).await;
let ready = ws.recv_json().await;
assert_eq!(ready["type"], "ready");
// Send speak
ws.send_json(json!({"type": "speak", "text": "Hello."})).await;
// Collect audio chunks
let mut chunks = vec![];
loop {
let msg = ws.recv().await;
match msg {
WsMsg::Binary(data) => chunks.push(data),
WsMsg::Text(json) => {
let data: Value = serde_json::from_str(&json).unwrap();
if data["type"] == "complete" { break; }
}
}
}
assert!(!chunks.is_empty());
}
```
### 7.3 Performance Targets
| Metric | Target | Acceptable | Warning |
|--------|--------|------------|---------|
| First Audio (short) | < 150ms | < 200ms | > 300ms |
| First Audio (medium) | < 200ms | < 300ms | > 500ms |
| First Audio (long) | < 300ms | < 500ms | > 800ms |
| Inter-chunk | < 30ms | < 50ms | > 100ms |
| Memory (GPU) | < 4GB | < 6GB | > 8GB |
| Memory (CPU) | < 2GB | < 4GB | > 8GB |
---
## 8. Implementation Phases
### Phase 1: Candle-Based Qwen3-TTS Module (Week 1-2)
**Deliverables:**
- [ ] `makima/src/tts/mod.rs` — TTS trait + factory
- [ ] `makima/src/tts/chatterbox.rs` — Move existing code from tts.rs
- [ ] `makima/src/tts/qwen3/model.rs` — 28-layer LM backbone (extend candle Qwen2)
- [ ] `makima/src/tts/qwen3/code_predictor.rs` — MTP module (5 layers, 16 codebooks)
- [ ] `makima/src/tts/qwen3/speech_tokenizer.rs` — ConvNet encoder/decoder + RVQ
- [ ] `makima/src/tts/qwen3/config.rs` — Config from safetensors
- [ ] `makima/src/tts/qwen3/generate.rs` — Autoregressive generation with KV cache
- [ ] Add `candle-core`, `candle-nn`, `candle-transformers` to Cargo.toml
**Success Criteria:**
- Model loads safetensors weights successfully
- Can generate audio from text via direct inference
- First audio latency < 500ms (initial, unoptimized)
### Phase 2: WebSocket Handler + Voice Assets (Week 2-3)
**Deliverables:**
- [ ] Update `makima/src/server/handlers/speak.rs` — Direct TTS handler (no proxy)
- [ ] Lazy model loading via `SharedState`
- [ ] Voice prompt caching
- [ ] Makima voice asset acquisition and processing
- [ ] Basic error handling and session management
**Success Criteria:**
- `/api/v1/speak` endpoint produces streaming audio
- Default Makima voice works
- Error handling matches specification
### Phase 3: Optimization + Integration (Week 3-4)
**Deliverables:**
- [ ] Streaming audio generation (token-by-token decoding)
- [ ] GPU memory optimization
- [ ] Listen page integration for bidirectional speech
- [ ] Session coordination between STT and TTS
- [ ] Full test suite (unit, integration)
- [ ] Latency benchmarks
**Success Criteria:**
- First audio latency < 200ms
- Memory usage < 6GB
- All tests passing
- Documentation complete
---
## Appendix
### A. Dependencies
#### Rust (Cargo.toml additions)
```toml
[dependencies]
candle-core = "0.8"
candle-nn = "0.8"
candle-transformers = "0.8"
# Keep existing: tokenizers, hf-hub, ndarray (for compatibility)
```
### B. Environment Variables
```bash
# TTS Configuration
TTS_ENGINE=qwen3 # "qwen3" or "chatterbox"
TTS_MODEL_ID=Qwen/Qwen3-TTS-12Hz-0.6B-Base
TTS_DEVICE=cuda:0 # "cuda:0", "metal", or "cpu"
TTS_VOICES_DIR=models/voices
TTS_DEFAULT_VOICE=makima
```
### C. References
1. [Qwen3-TTS-12Hz-0.6B-Base (Hugging Face)](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-0.6B-Base)
2. [Qwen3-TTS GitHub Repository](https://github.com/QwenLM/Qwen3-TTS)
3. [Qwen3-TTS Technical Report (arXiv:2601.15621)](https://arxiv.org/abs/2601.15621)
4. [Candle — HuggingFace Rust ML Framework](https://github.com/huggingface/candle)
5. [axum WebSocket Documentation](https://docs.rs/axum/latest/axum/extract/ws/index.html)
6. [docs/research/rust-native-tts-research.md](../research/rust-native-tts-research.md) — Detailed feasibility analysis
### D. Glossary
| Term | Definition |
|------|------------|
| **TTS** | Text-to-Speech: Converting text input to audio output |
| **STT** | Speech-to-Text: Converting audio input to text output |
| **Voice Cloning** | Creating synthetic speech that mimics a specific speaker |
| **Voice Prompt** | Pre-computed speaker embedding for voice cloning |
| **Candle** | HuggingFace's minimalist Rust ML framework |
| **SafeTensors** | Efficient, safe model weight serialization format |
| **RVQ** | Residual Vector Quantization — multi-codebook audio tokenization |
| **MTP** | Multi-Token Prediction — code predictor generating 16 codebook layers |
| **bf16** | Brain floating-point 16-bit format for GPU computation |
