Smart Dubbing Studio

Solving Alignment and Subtitle Sync in Video Dubbing

Producing Chinese tutorials or videos often requires tedious manual work: aligning subtitle timestamps, managing TTS sentence breaks, and fixing audio-video desynchronization when the dubbed track is shorter than the original footage. smart-dubbing-studio wraps these tasks into an automated pipeline, using AI-guided conversation to handle environment checks, engine configuration, and execution without requiring manual script writing.

Core Capabilities and Workflow

• Dual TTS Engine Support: Compatible with iFlytek spark (providing word-level timestamps and multiple voices) and edge-tts (a keyless fallback). The AI guides the selection and maps the appropriate vcn or voice parameters.
• Granular Template Control: Parses MM:SS,FF frame-level timecodes. Users can mark filler segments (by leaving text blank) to retain original audio, use (静音) for silence, force TTS pauses with backticks ` (invisible in subtitles), or override specific pronunciations using {tts:...}.
• Frame-by-Frame Editor: Includes a local subtitle-editor.html tool where F marks segment ends and G creates gaps. It supports Ctrl+Z undo and localStorage caching, eliminating the pain of manually typing timecodes while watching the video.
• V3.7 Upgrades: Automatically splits long text into "read-one-show-one" short SRT sentences based on word-level timestamps. If the dubbed audio is 2% shorter than the video, it automatically applies ffmpeg setpts and atempo filters for up to 1.5x speed-up, maintaining audio-video sync.

Boundaries and Caveats

• SRT Splitting Depends on Timestamps: The iFlytek spark engine fully supports per-sentence SRT splitting, whereas edge-tts defaults to showing the entire paragraph as a single subtitle.
• Environment Dependencies: Local ffmpeg and Python 3.7+ are required. Python packages like websocket-client and edge-tts are installed automatically upon first run (with fallback to domestic mirrors).
• Credential Security: Using iFlytek requires configuring APPID and APIKey in tts_config.json. Always remove this file or replace values with placeholders before sharing projects to prevent sensitive data leaks.