AI Multilingual Dubbing

Engineering the Multilingual Voiceover Workflow

Creating voiceover audio for multiple markets (e.g., courses, ads, product demos) presents several concrete engineering hurdles:
- Disorganized Script & Locale Management: Tracking script status, terminology, pronunciation notes, and duration targets across dozens of languages becomes unwieldy.
- High-Cost Voice Selection: Evaluating voice compatibility across languages typically requires manual, per-language audition cycles, which is inefficient.
- Opaque Billing & Scope Creep: With per-request TTS billing, final costs are hard to estimate without a disciplined confirmation process.
- Complex Async Task Handling: Generation tasks can be long-running, involving submission, polling, timeout recovery, and result aggregation—error-prone when managed manually.

Core Workflow & Capabilities

This skill addresses these issues through a structured process centered around a segment × locale production matrix. The workflow follows these key phases:

1. Matrix-Driven Script Preparation

• Source content (text or audio/video transcripts) is divided into natural segments (e.g., one ad unit or chapter), never splitting sentences.
• For each segment-target locale pair, a matrix cell is created, mandating fields like the BCP-47 tag, confirmed copy, protected terms, voice, delivery direction, format, and duration target. This ensures atomic, traceable information.

2. Voice Discovery & Trial Strategy

• It reuses voices already accepted in the dialogue. If unavailable, it calls beatra.voices.list for filtering and recommends 2-3 candidates.
• Adheres to a trial-before-pay principle: Voice selection uses only free audition URLs, incurring no synthesis fees.
• For multilingual scopes, a high-risk representative segment (containing names, numbers, terminology, or CTAs) is chosen per language for a trial render. Defaults use model: "auto" and speed: 1.0.

3. Scope Confirmation & Controlled Execution

• Before initiating any beatra.speech.synthesize paid operation, the complete paid scope is presented in a readable matrix, explicitly stating each cell will incur one paid render.
• Upon a single user confirmation of the entire matrix, the skill creates a unique client_request_id per cell and submits tasks. Task status is polled via beatra.tasks.get, respecting deadline_at, and proactively handles recovery without infinite waits.
• All MCP operations are strictly executed via the bundled scripts/mcp_client.py, avoiding dependency on the host connector.

4. Result Delivery & Verification

• After success, results are delivered grouped by locale, including the audio URL, asset ID, MIME type, actual sample rate, and duration.
• The skill does not claim to perform translation, subtitles, lip-sync, or precise duration matching. Modifications require re-confirmation and re-rendering only for affected cells, leaving accepted content unchanged.

Application Scope & Critical Constraints

This skill is focused on unidirectional text-to-speech generation and management, suitable for:
- Batch audio production from finalized multilingual scripts.
- Projects requiring strict cost control and state tracking.

Key boundaries to note:
- Input Must Be Text: For video/audio sources, transcripts must first be obtained and always treated as draft until user confirmation.
- Out-of-Scope Work: Does not provide copy translation, lip-sync, audio/video editing, or frame-accurate duration matching.
- Billing & State Dependencies: Paid operations require explicit matrix confirmation. Task execution is asynchronous, requiring adherence to its polling and recovery mechanisms.
- Automatic Updates: The skill may silently check for official updates, replacing only its own files without affecting the user's in-progress commands.