summaryrefslogtreecommitdiff
path: root/makima/src/server
Commit message (Collapse)AuthorAgeFilesLines
* Rerun plan when directive goal is edited (#61)soryu2026-02-131-1/+22
| | | | | | | | | | | | | | | When a directive's goal is updated, pending/ready/failed/skipped steps are now automatically cleared so that replanning generates fresh steps aligned with the new goal. The planning prompt is also improved to clearly categorize existing steps by status and provide explicit instructions for re-evaluation. Changes: - Add clear_pending_directive_steps() repository function to remove non-started steps when the goal changes - Call step cleanup in the update_goal HTTP handler - Restructure the planning prompt to categorize steps (completed, running, pending, failed, skipped) with clear instructions for each category Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Fix worktree branching for directive tasks and remove memoriessoryu2026-02-133-409/+8
|
* Add task cleanup and directive PR updatingsoryu2026-02-123-0/+5
|
* Add depends_on for directive taskssoryu2026-02-121-5/+67
| | | | | # Conflicts: # makima/src/orchestration/directive.rs
* makima: Add an optional memory system for directives (#59)soryu2026-02-123-9/+411
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * feat: makima: Add an optional memory system for directives: Add directive_memories database table and migration * feat: makima: Add an optional memory system for directives: Update directive skill documentation with memory commands * feat: makima: Add an optional memory system for directives: Add repository functions for directive memory CRUD * feat: makima: Add an optional memory system for directives: Add frontend API functions and types for directive memory * feat: makima: Add an optional memory system for directives: Add Rust models for directive memory * WIP: heartbeat checkpoint * WIP: heartbeat checkpoint * WIP: heartbeat checkpoint * WIP: heartbeat checkpoint * feat: makima: Add an optional memory system for directives: Add memory panel to frontend DirectiveDetail component * Merge remote-tracking branch 'origin/makima/makima--add-an-optional-memory-system-for-directiv-5de1e06d' into combined branch * Merge remote-tracking branch 'origin/makima/makima--add-an-optional-memory-system-for-directiv-c8298c6c' into combined branch * feat: makima: Add an optional memory system for directives: Create useMultiTaskSubscription hook for multi-output WebSocket streaming * feat: makima: Add an optional memory system for directives: Create DirectiveLogStream component for stern-like multi-task output viewing * feat: makima: Add an optional memory system for directives: Integrate log stream panel into directive detail page
* Add auto-PR creation for remote repos in contractssoryu2026-02-091-4/+104
|
* Set directive env vars correctly for daemonsoryu2026-02-095-0/+14
|
* Add directive task progressionsoryu2026-02-091-0/+9
|
* Add directive initsoryu2026-02-091-0/+19
|
* Resume contracts from patchessoryu2026-02-093-0/+100
|
* Add new directive mechanism v3soryu2026-02-0911-26/+945
|
* Remove directive mechanismsoryu2026-02-086-869/+8
|
* Fix directive evaluation and add to frontendsoryu2026-02-081-2/+2
|
* Fixes for directive chain initsoryu2026-02-083-2/+64
|
* Check on completion for contractssoryu2026-02-071-1/+18
|
* Add directive monitor contractssoryu2026-02-073-10/+233
|
* Show directive init on frontendsoryu2026-02-071-2/+19
|
* Add directive init mechanismsoryu2026-02-074-1/+70
|
* Add new directive initial implementationsoryu2026-02-074-11/+486
|
* Remove directives for reimplementationsoryu2026-02-075-2272/+3
|
* Fix: Link directives and contractssoryu2026-02-061-1/+67
|
* Fix: Cleanup old chain codesoryu2026-02-063-1646/+489
|
* Fix: Directives fixessoryu2026-02-061-0/+15
|
* Fix: Directives APIsoryu2026-02-062-2/+93
|
* Add directive-first chain system redesignsoryu2026-02-055-1304/+1622
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Redesigns the chain system with a directive-first architecture where Directive is the top-level entity (the "why/what") and Chains are generated execution plans (the "how") that can be dynamically modified. Backend: - Add database migration for directive system tables - Add Directive, DirectiveChain, ChainStep, DirectiveEvent models - Add DirectiveVerifier and DirectiveApproval models - Add orchestration module with engine, planner, and verifier - Add comprehensive API handlers for directives - Add daemon CLI commands for directive management - Add directive skill documentation - Integrate contract completion with directive engine - Add SSE endpoint for real-time directive events Frontend: - Add directives route with split-view layout - Add 6-tab detail view (Overview, Chain, Events, Evaluations, Approvals, Verifiers) - Add React Flow DAG visualization for chain steps - Add SSE subscription hook for real-time event updates - Add useDirectives and useDirectiveEventSubscription hooks - Add directive types and API functions Fixes: - Fix test failures in ws/protocol, task_output, completion_gate, patch - Fix word boundary matching in looks_like_task() - Fix parse_last() to find actual last completion gate - Fix create_export_patch when merge-base equals HEAD - Clean up clippy warnings in new code Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add makima directivessoryu2026-02-054-3/+1320
|
* Add multi-repository support for chainssoryu2026-02-052-4/+344
| | | | | | | | | | | | | | | | | | | | | | Chains can now have multiple repositories attached, with one marked as primary. Repositories are used by contracts created from chain definitions. Backend changes: - Add chain_repositories table migration - Add ChainRepository model with CRUD operations - Add API endpoints for listing, adding, deleting repositories - Add endpoint to set a repository as primary - Update Chain and ChainEditorData models to use repositories - Update chain parser to support repositories in YAML format - Remove deprecated repository_url/local_path from Chain Frontend changes: - Add ChainRepository interface and API functions - Add repository section to ChainEditor showing attached repos - Add modal for adding new repositories (remote or local) - Support setting primary repository and removing repositories Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Remove chain supervisor capabilitysoryu2026-02-041-132/+4
| | | | | | | | | | | | Chains no longer spawn a supervisor task. Checkpoint contracts will be automatically run as part of the DAG execution when dependencies complete. - Remove supervisor task creation from start_chain handler - Remove chain supervisor CLI commands - Remove supervisor_task_id from StartChainResponse - Remove withSupervisor option from frontend Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add chain checkpoint contractssoryu2026-02-042-9/+170
|
* Allow chain creation via web interfacesoryu2026-02-032-3/+663
|
* Add 'Discuss Contract' feature to listen page (#57)soryu2026-02-034-2/+604
|
* Add makima chain mechanismsoryu2026-02-033-1/+626
|
* Release in makima reposoryu2026-02-022-7/+5
| | | | Also remove all other TTS models
* Make makima more opinionated and structuredsoryu2026-02-0211-1250/+77
|
* Use chatterbox TTSsoryu2026-02-012-11/+10
|
* feat: Implement Phase 3.5 - Supervisor Status APImakima/contract-management-phase3soryu2026-02-012-0/+346
|\ | | | | | | | | | | | | | | | | | | - Add SupervisorStatusResponse for status endpoint - Add SupervisorHeartbeatEntry and history response types - Add SupervisorSyncResponse for sync endpoint - Add HeartbeatHistoryQuery for pagination - Resolve merge conflict keeping both API types and unit tests Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
| * feat: Add Supervisor Status API endpoints (Phase 3 Task 3.5)soryu2026-02-012-0/+346
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implement REST API endpoints for querying supervisor status: - GET /api/v1/contracts/{id}/supervisor/status Returns current supervisor status including task_id, state, phase, current_activity, progress, last_heartbeat, and pending_task_ids - GET /api/v1/contracts/{id}/supervisor/heartbeats?limit=10 Returns paginated supervisor activity history from history_events - POST /api/v1/contracts/{id}/supervisor/sync Triggers a sync to refresh the supervisor's last_activity timestamp New types added: - SupervisorStatusResponse - Status endpoint response - SupervisorHeartbeatEntry - Individual heartbeat history entry - SupervisorHeartbeatHistoryResponse - Heartbeat history with pagination - SupervisorSyncResponse - Sync endpoint response - HeartbeatHistoryQuery - Query params for heartbeats endpoint Repository helpers: - get_supervisor_status() - Combined info from supervisor_states and tasks - get_supervisor_activity_history() - Activity timeline from history_events - count_supervisor_activity_history() - Total count for pagination - sync_supervisor_state() - Refresh last_activity timestamp Error handling: - 404 for contract not found (CONTRACT_NOT_FOUND) - 404 for no supervisor (SUPERVISOR_NOT_FOUND) - Proper fallback when supervisor_state record doesn't exist but task does Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* | feat: Implement Phase 3 Tasks 3.3 and 3.4 - Supervisor State Persistence and ↵soryu2026-02-013-25/+595
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Restoration Task 3.3: Supervisor State Persistence - Add migration 20260201000001_enhanced_supervisor_state.sql with new columns: - state (supervisor state enum) - current_activity (description) - progress (0-100) - error_message (for failed states) - spawned_task_ids (tasks created by supervisor) - pending_questions (questions awaiting user response) - restoration_count, last_restored_at, restoration_source (restoration tracking) - Update SupervisorState model with new fields - Add PendingQuestion struct for tracking unanswered questions - Add SupervisorRestorationContext for returning restoration info - Add StateValidationResult and StateRecoveryAction for state validation State persistence functions in repository.rs: - update_supervisor_detailed_state() - Update state, activity, progress - add_supervisor_spawned_task() - Track spawned tasks - add_supervisor_pending_question() - Track pending questions - remove_supervisor_pending_question() - Clear answered questions - save_supervisor_state_full() - Full state save (UPSERT) - mark_supervisor_restored() - Increment restoration count - get_supervisors_with_pending_questions() - Find supervisors with pending questions - get_supervisor_state_for_restoration() - Load state for restoration - validate_spawned_tasks() - Validate task consistency - update_supervisor_phase() - Update on phase change - update_supervisor_heartbeat_state() - Lightweight heartbeat update State save points: - On task spawn (save_state_on_task_spawn) - On question asked (save_state_on_question_asked) - On question answered (clear_pending_question) - On phase change (save_state_on_phase_change) - On heartbeat (update_supervisor_heartbeat_state) Task 3.4: Supervisor Restoration Protocol - Add restoration detection when supervisor starts with existing state - Implement validate_supervisor_state() for state consistency checks - Implement restore_supervisor() with validation and context generation - Add redeliver_pending_questions() for re-delivering questions after crash - Add generate_restoration_context_message() for Claude context injection - Update resume_supervisor endpoint to return RestorationInfo - Re-deliver pending questions when supervisor resumes Restoration flow: 1. Daemon restarts or task reassigned 2. Load supervisor state from supervisor_states 3. If NOT FOUND: Start fresh, log warning 4. If FOUND: Validate state consistency 5. If INVALID: Start from last checkpoint 6. If VALID: Restore conversation history 7. Check for pending questions - re-deliver to user 8. Check for waiting tasks - resume waiting state 9. Send restoration context to Claude 10. Resume execution from last state Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* | feat: Implement Phase 3 Tasks 3.1 and 3.2 - SupervisorState enum and ↵soryu2026-02-011-0/+83
|/ | | | | | | | | | | | | | | | | | | | | | | | | Heartbeat Infrastructure Task 3.1: Enhanced Supervisor State Enum - Add SupervisorStateEnum with states: Initializing, Idle, Working, WaitingForUser, WaitingForTasks, Blocked, Completed, Failed, Interrupted - Implement Display, FromStr, Default, and serde serialization - Add SupervisorHeartbeatRecord and SupervisorHeartbeatRequest structs Task 3.2: Heartbeat Infrastructure - Create supervisor_heartbeats migration with proper indexes and constraints - Add heartbeat storage functions to repository.rs: - create_supervisor_heartbeat - get_latest_supervisor_heartbeat - get_supervisor_heartbeats - get_contract_supervisor_heartbeats - cleanup_old_heartbeats (24 hour TTL support) - find_stale_supervisors (for dead supervisor detection) - Add SupervisorHeartbeat message to protocol.rs with enhanced fields - Update mesh_daemon.rs to process and store supervisor heartbeats - Add unit tests for SupervisorStateEnum and heartbeat serialization Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: Add contract management system improvements (Phase 1)makima/contract-management-improvementssoryu2026-01-312-21/+327
| | | | | | | | | | | | | | - Add docs/contract-management-spec.md with full system design - Add docs/plans/implementation-plan.md with 5-phase rollout plan - Add validate_deliverable() function and use in mark_deliverable_complete - Add PhaseChangeResult enum and change_contract_phase_with_version() with FOR UPDATE locking - Enforce phase_guard at API level for all callers This addresses critical issues in contract management: - Deliverable validation to prevent marking non-existent deliverables complete - Version conflict detection for phase changes with row locking - Phase guard enforcement at API level (applies to all callers including supervisors) - Comprehensive specification and implementation plan for future phases
* Add auto_merge_local option for local-only contracts (#50)soryu2026-01-317-30/+51
| | | | | | | | | | | | | | | | | | | | When local_only=true on a contract, all completion actions are skipped. This adds a new option auto_merge_local that, when enabled along with local_only, will automatically merge completed task changes to the master/main branch locally (without pushing or creating PRs). Changes: - Add auto_merge_local column to contracts table (migration) - Add auto_merge_local field to Contract model and summary - Update CreateContractRequest and UpdateContractRequest structs - Update contract repository create/update functions - Add auto_merge_local to WebSocket protocol StartTask command - Pass auto_merge_local through spawn_task and run_task functions - Modify task manager completion logic: if local_only=true AND auto_merge_local=true, execute 'merge' completion action locally - Update all server handlers to retrieve and pass auto_merge_local - Add TypeScript types to frontend components Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Add autodetection of master for PR creationsoryu2026-01-292-6/+4
|
* Fix Red Team UI visibility by adding red_team_enabled to ContractSummarysoryu2026-01-291-0/+4
| | | | | | | | | | | | | | | | | The Red Team toggle was implemented in the frontend but not visible because the backend API's ContractSummary response struct was missing the red_team_enabled field. The frontend relies on this field to: 1. Show the red team badge in the contract list view 2. Show the red team badge and tab in the contract detail view Changes: - Add red_team_enabled field to ContractSummary struct in models.rs - Update list_contracts_for_owner SQL query to include red_team_enabled - Update get_contract_summary_for_owner SQL query to include red_team_enabled - Update all fallback ContractSummary constructions in contracts.rs handler Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fix model loading for TTS / speaksoryu2026-01-291-2/+10
|
* Fix contract type selectionsoryu2026-01-294-5/+428
|
* Fix worktree cleanup to not run for shared worktreessoryu2026-01-296-0/+29
|
* Fix makima supervisor pr CLI commandsoryu2026-01-292-22/+23
|
* Replace TTS endpoint with Rust-native Qwen3-TTS (#41)soryu2026-01-283-7/+323
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * chore: fix unused import warnings in qwen3-tts module - Remove unused import 'IndexOp' in model.rs - Remove unused import 'DType' in speech_tokenizer.rs - Add #[allow(dead_code)] to codebook_dim field in RvqCodebook Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add voice loading and selection for TTS cloning Add voice reference audio loading so the TTS speak handler can perform voice cloning using reference WAV files from the voices/ directory. - Add voice.rs module: loads manifest.json and reference.wav for a given voice_id, decodes via symphonia, resamples to 24kHz for the TTS engine - Update speak.rs: resolve voice_id from the speak request (default "makima"), load reference audio, pass it to engine.generate() - Add voices/makima/README.md with instructions for obtaining reference audio (extraction from YouTube, recording, ffmpeg conversion) - Graceful fallback: if reference audio is missing, TTS proceeds without voice cloning using the model's default voice Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * [WIP] Heartbeat checkpoint - 2026-01-28 03:49:13 UTC --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Add Qwen3-TTS streaming endpoint for voice synthesis (#40)soryu2026-01-285-1/+460
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Task completion checkpoint * Task completion checkpoint * Task completion checkpoint * Add Qwen3-TTS research document for live TTS replacement Research findings for replacing Chatterbox TTS with Qwen3-TTS-12Hz-0.6B-Base: - Current TTS: Chatterbox-Turbo-ONNX with batch-only generation, no streaming - Qwen3-TTS: 97ms end-to-end latency, streaming support, 3-second voice cloning - Voice cloning: Requires 3s reference audio + transcript (Makima voice planned) - Integration: Python service with WebSocket bridge (no ONNX export available) - Languages: 10 supported including English and Japanese Document includes: - Current architecture analysis (makima/src/tts.rs) - Qwen3-TTS capabilities and requirements - Feasibility assessment for live/streaming TTS - Audio clip requirements for voice cloning - Preliminary technical approach with architecture diagrams Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * [WIP] Heartbeat checkpoint - 2026-01-27 03:11:15 UTC * Add Qwen3-TTS research documentation Comprehensive research on replacing Chatterbox TTS with Qwen3-TTS-12Hz-0.6B-Base: - Current TTS implementation analysis (Chatterbox-Turbo-ONNX in makima/src/tts.rs) - Qwen3-TTS capabilities: 97ms streaming latency, voice cloning with 3s reference - Cross-lingual support: Japanese voice (Makima/Tomori Kusunoki) speaking English - Python microservice architecture recommendation (FastAPI + WebSocket) - Implementation phases and technical approach - Hardware requirements and dependencies Key findings: - Live/streaming TTS is highly feasible with 97ms latency - Voice cloning fully supported with 0.95 speaker similarity - Recommended: Python microservice with WebSocket streaming Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Add comprehensive Qwen3-TTS integration specification This specification document defines the complete integration of Qwen3-TTS-12Hz-0.6B-Base as a replacement for the existing Chatterbox-Turbo TTS implementation. The document covers: ## Functional Requirements - WebSocket endpoint /api/v1/speak for streaming TTS - Voice cloning with default Makima voice (Japanese VA speaking English) - Support for custom voice references - Detailed client-to-server and server-to-client message protocols - Integration with Listen page for bidirectional speech ## Non-Functional Requirements - Latency targets: < 200ms first audio byte - Audio quality: 24kHz, mono, PCM16/PCM32f - Hardware requirements: CUDA GPU with 4-8GB VRAM - Scalability: 10 concurrent sessions per GPU ## Architecture Specification - Python TTS microservice with FastAPI/WebSocket - Rust proxy endpoint in makima server - Voice prompt caching mechanism (LRU cache) - Error handling and recovery strategies ## API Contract - Complete WebSocket message format definitions (TypeScript) - Error codes and responses (TTS_UNAVAILABLE, SYNTHESIS_ERROR, etc.) - Session state machine and lifecycle management ## Voice Asset Requirements - Makima voice clip specifications (5-10s WAV, transcript required) - Storage location: models/voices/makima/ - Metadata format for voice management ## Testing Strategy - Unit tests for Python TTS service and Rust proxy - Integration tests for WebSocket flow - Latency benchmarks with performance targets - Test data fixtures for various text lengths Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Add Qwen3-TTS implementation plan Comprehensive implementation plan for replacing Chatterbox-TTS with Qwen3-TTS streaming TTS service, including: - Task breakdown with estimated hours for each phase - Phase 1: Python TTS microservice (FastAPI, WebSocket) - Phase 2: Rust proxy integration (speak.rs, tts_client.rs) - Detailed file changes and new module structure - Testing plan with unit, integration, and latency benchmarks - Risk assessment with mitigation strategies - Success criteria for each phase Based on specification in docs/specs/qwen3-tts-spec.md Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Add author and research references to TTS implementation plan Add links to research documentation and author attribution. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * [WIP] Heartbeat checkpoint - 2026-01-27 03:25:06 UTC * Add Python TTS service project structure (Phase 1.1-1.3) Create the initial makima-tts Python service directory structure with: - pyproject.toml with FastAPI, Qwen-TTS, and torch dependencies - config.py with pydantic-settings TTSConfig class - models.py with Pydantic message models (Start, Speak, Stop, Ready, etc.) This implements tasks P1.1, P1.2, and P1.3 from the Qwen3-TTS implementation plan. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Add TTS engine and voice manager for Qwen3-TTS (Phase 1.4-1.5) Implement core TTS functionality: - tts_engine.py: Qwen3-TTS wrapper with streaming audio chunk generation - voice_manager.py: Voice prompt caching with LRU eviction and TTL support Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * [WIP] Heartbeat checkpoint - 2026-01-27 03:30:06 UTC * Add TTS proxy client and message types (Phase 2.1, 2.2, 2.4) - Add tts_client.rs with TtsConfig, TtsCircuitBreaker, TtsError, TtsProxyClient, and TtsConnection structs for WebSocket proxying - Add TTS message types to messages.rs (TtsAudioEncoding, TtsPriority, TtsStartMessage, TtsSpeakMessage, TtsStopMessage, TtsClientMessage, TtsReadyMessage, TtsAudioChunkMessage, TtsCompleteMessage, TtsErrorMessage, TtsStoppedMessage, TtsServerMessage) - Export tts_client module from server mod.rs - tokio-tungstenite already present in Cargo.toml Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Add TTS WebSocket handler and route (Phase 2.3, 2.5, 2.6) - Create speak.rs WebSocket handler that proxies to Python TTS service - Add TtsState fields (tts_client, tts_config) to AppState - Add with_tts() builder and is_tts_healthy() methods to AppState - Register /api/v1/speak route in the router - Add speak module export in handlers/mod.rs The handler forwards WebSocket messages bidirectionally between the client and the Python TTS microservice with proper error handling. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Add Makima voice profile assets for TTS voice cloning Creates the voice assets directory structure with: - manifest.json containing voice configuration (voice_id, speaker, language, reference audio path, and Japanese transcript placeholder) - README.md with instructions for obtaining voice reference audio Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Add Rust-native Qwen3-TTS integration research document Research findings for integrating Qwen3-TTS-12Hz-0.6B-Base directly into the makima Rust codebase without Python. Key conclusions: - ONNX export is not viable (unsupported architecture) - Candle (HF Rust ML framework) is the recommended approach - Model weights available in safetensors format (2.52GB total) - Three components needed: LM backbone, code predictor, speech tokenizer - Crane project has Qwen3-TTS as highest priority (potential upstream) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * [WIP] Heartbeat checkpoint - 2026-01-27 11:21:43 UTC * [WIP] Heartbeat checkpoint - 2026-01-27 11:24:19 UTC * [WIP] Heartbeat checkpoint - 2026-01-27 11:26:43 UTC * feat: implement Rust-native Qwen3-TTS using candle framework Replace monolithic tts.rs with modular tts/ directory structure: - tts/mod.rs: TtsEngine trait, TtsEngineFactory, shared types (AudioChunk, TtsError), and utility functions (save_wav, resample, argmax) - tts/chatterbox.rs: existing ONNX-based ChatterboxTTS adapted to implement TtsEngine trait with Mutex-wrapped sessions for Send+Sync - tts/qwen3/mod.rs: Qwen3Tts entry point with HuggingFace model loading - tts/qwen3/config.rs: Qwen3TtsConfig parsing from HF config.json - tts/qwen3/model.rs: 28-layer Qwen3 transformer with RoPE, GQA (16 heads, 8 KV heads), SiLU MLP, RMS norm, and KV cache - tts/qwen3/code_predictor.rs: 5-layer MTP module predicting 16 codebooks - tts/qwen3/speech_tokenizer.rs: ConvNet encoder/decoder with 16-layer RVQ - tts/qwen3/generate.rs: autoregressive generation loop with streaming support Add candle-core, candle-nn, candle-transformers, safetensors to Cargo.toml. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: integrate TTS engine into speak WebSocket handler - Update speak.rs handler to use TTS engine directly from SharedState instead of returning a stub "not implemented" error - Add TtsEngine (OnceCell lazy-loaded) to AppState in state.rs with get_tts_engine() method for lazy initialization on first connection - Implement full WebSocket protocol: client sends JSON speak/cancel/stop messages, server streams binary PCM audio chunks and audio_end signals - Create voices/makima/manifest.json for Makima voice profile configuration - All files compile successfully with zero errors Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add /speak TTS page with WebSocket audio playback Add a new /speak frontend page for text-to-speech via WebSocket. The page accepts text input and streams synthesized PCM audio through the Web Audio API. Includes model loading indicator, cancel support, and connection status. Also adds a loading bar to the listen page ControlPanel during WebSocket connection. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Add patch merging and fix task healthcheck failing due to worktreessoryu2026-01-272-25/+138
|
generated by cgit v1.2.3 (git 2.39.1) at 2026-05-21 09:32:25 +0000