Video Article Link Auto-Archival Pipeline

Problem

Engineers often send Douyin, Bilibili, Xiaohongshu, WeChat, and web links to an agent and expect usable local media or searchable text. The hard part is not simply “save it,” but the uneven platform behavior: yt-dlp may work for some sites, while Douyin and Xiaohongshu need browser context, mobile headers, and response inspection. Videos also need transcription, and articles need topic routing. Without a stable pipeline, downloads fail, technical terms are mistranscribed, and content lands in the wrong knowledge base.

How It Works

The skill follows identify → process → clean → route. It first detects the link type by domain: douyin.com and bilibili.com go to the video path; mp.weixin.qq.com and general web pages go to the article path; Xiaohongshu content is routed based on whether it is image-text or video. For open platforms such as Bilibili, the video path can use yt-dlp. For Douyin and Xiaohongshu, it opens a mobile page with playwright, extracts or listens for the video source, then converts audio with ffmpeg and transcribes with Whisper. If an independent subtitle stream exists, it cross-checks the Whisper output against the subtitles; otherwise it flags product names and dense technical terms for review. A DeepSeek V4 cleaning pass merges the verified results. The article path extracts the article body, then routes it using topic rules in config.yaml; if no rule matches, it asks the user instead of guessing the destination.

Limits

This is closer to a link ingestion and knowledge routing workflow than a general media server. Douyin and Xiaohongshu handling depends on browser environment and request behavior, long videos should be segmented before transcription, and final storage depends on routing rules and user confirmation.