Douyin Video Text Extraction

The Problem It Solves

In scenarios like content creation, data analysis, and information archiving, there is a frequent need to convert spoken content from Douyin videos or local audio/video files into editable, searchable text. Manual transcription is time-consuming and error-prone, especially for long videos or low-quality recordings. This skill automates the entire pipeline from extraction to structured output.

Workflow & Core Capabilities

The skill operates in two primary modes, sharing the same transcription and output logic.

1. Douyin Link Mode
- Video Extraction: When a user provides a Douyin share link or command text, the skill uses Playwright to open the short link in a headless Chrome browser. It waits for the JavaScript to load and captures the video.src directly from the DOM, then downloads the original video. This method works without login and bypasses common login modal interference.
- Key Advantage: It employs DOM parsing instead of network interception, ensuring stable extraction.

2. Local File Mode
- Direct Read: Users can directly specify a local media file path (absolute, relative, or attachment reference). The skill does not move or copy the source file; it reads it directly.
- Broad Format Support: Compatible with a wide range of media formats including .mp4, .mp3, .wav, .flac, etc.

Universal Core Steps: Transcription and Agent Refinement
- Local Transcription: Uses the Whisper medium model (~1.4GB, auto-downloaded and cached on first run) to perform speech recognition on the extracted or specified media, generating a raw transcript with timestamps. The script includes auto-repair for corrupted model caches.
- Agent Refinement: After transcription, the Agent itself performs post-processing on the raw text. This step does not invoke any external LLM and strictly adheres to rules:
- Allowed: Adding necessary punctuation, correcting obvious ASR (Automatic Speech Recognition) typos, and semantic segmentation.
- Prohibited: Deleting original content, rewriting or summarizing, adding commentary or reviews. This ensures the output text is faithful to the original speech.
- Structured Output: The final output is a clean, readable .md file, while the raw .txt and .json transcripts are preserved for verification.

Limitations and Notes

• Environment Dependency: Requires a local Python 3.10–3.12 environment. Dependencies (including PyTorch, Whisper, etc.) must be installed in an isolated virtual environment via the setup_env.py script. This script must be re-run after migrating to a new machine.
• Resource Usage: The initial run downloads ~4GB of models and dependencies and requires sufficient disk space (~4GB).
• Content Processing Boundary: The Agent's refinement is limited to text polishing and correction; it does not perform any content-level additions or deletions. This is a deliberate design constraint, not a flaw.
• Platform Dynamics: The extraction of Douyin links relies on the page's DOM structure. Updates to the platform's frontend could potentially disrupt extraction, but the skill is designed with auto-retry and fallback mechanisms.