Introduction

In DeepSeek Harness (DSH), when asked about video content, what the Agent needs is the visual frames.

Existing solutions fall into two basic categories: using a visual bridge plugin (sending frames to a cloud VLM, returning only textual conclusions, requiring an API key, and producing secondhand conclusions), or having the Agent assemble ffmpeg commands itself (requiring timestamp and edge-case handling, prone to errors).

dsh-video-see offers a third path: no external VLM calls, no API key, and frames never leave the machine. It uses ffmpeg to extract frames, returns them to the Agent as real images, and includes timestamps.

What This Is

dsh-video-see is a plugin for DeepSeek Harness. It uses ffmpeg to extract video frames, returns real image content blocks, and marks timestamps. The author is HongtaoWang75, and the license is MIT.

Core Functions

  • Extracts video frames using ffmpeg.
  • Returns real images to the Agent.
  • Marks each frame with a timestamp.
  • No visual API or API key is required.
  • All processing is performed locally and never leaves the machine.
  • Supports multiple parameters: input, times, every, count, start, end, width, mode.
  • Can be configured through cordis.patch.yml: ffmpegPath, ffprobePath, maxFrames, maxWidth, maxPixels, quality, timeoutMs, toolTimeoutMs, graceMs.
  • Includes unit tests and contract validation scripts.

Installation and Enablement

  1. Run the installation command:
dsh plugin --profile web add dsh-video-see
  1. Restart dsh web (the plugin tree is composed when the process starts).
    Prerequisites: ffmpeg / ffprobe must be installed (or their absolute paths specified in the configuration). The plugin has zero runtime dependencies, no build step, and no network access.

Typical Usage

After installation, simply ask a question. The Agent will call video_see.
Examples:
- “What happened in D:\videos\demo.mp4?”
- “What does the small line of text on the screen at the 14-second mark say?”
- “How many shots has this video been cut into?”

Main parameters:
- input: video path (required).
- times: fixed-second values or a clock string; highest priority.
- every: one frame every N seconds.
- count: number of evenly divided frames to extract (1-64, default 8).
- start / end: specify the time range.
- width: upper limit for single-frame width.
- mode: frames (one image per frame) or sheet (combined into a single image sheet).

Applicable Scenarios and Notes

  • No audio transcription: audio is not transcribed; hasAudio only reports the presence of an audio track.
  • Does not go through ctx.fs: the input path is passed directly to ffmpeg, so file sandboxing does not apply to this read.
  • Discrete snapshots: frames are discrete, and motion or fleeting details may be missed.
  • ffmpeg dependency: ffmpeg must be installed.
  • Edge-case handling: automatically handles frame boundaries, rotation detection, portrait scaling by area, and scaling only when necessary.
  • Temporary files: temporary frames are deleted immediately.
  • Error messages: user-friendly failure messages are provided.

Short Conclusion

This plugin returns video frames to the DSH Agent as real images. Combined with the native visual capabilities of local models, it avoids secondary translation and API costs. For details, see the catalog page or GitHub repository.