Preface¶

DeepSeek Harness (dsh) is an intelligent agent runtime open-sourced by DeepSeek, officially positioned as a developer preview version, with the slogan “Everything is a plugin”: model adaptation, tool registration, session logging, and Agent loops can all be replaced by plugins without modifying the runtime source code. The official entry point for launching the Web UI is:

npx @deepseek-ai/dsh web

A more common gap on the coding agent side is the other side: mainstream dialogue models like DeepSeek and GLM are pure text-based and cannot see images. Error interfaces, design drafts, long chat screenshots, and front-end rendering results often have to be dictated first before letting the model guess. The community directory deepseek-harness-plugin.com is an independent site, not officially affiliated with DeepSeek / Magic Square; it categorizes and collects such extensions, among which there is a featured plugin labeled under “Tools and Capabilities”: agent-vision-toolkit.

This article is organized after cross-checking with the directory detail page, GitHub repository README / README_CN / AGENT_INSTALL.md, license, and official instructions of DeepSeek Harness: what it is, how the tools are divided, how commands are written, and what the relationship is with the native dsh access package.

What is it¶

agent-vision-toolkit is maintained by Anionex, with the repository address at Anionex/agent-vision-toolkit, licensed under MIT, and the main language is Python. The repository was created on 2026-08-01. The community directory categorizes it under “Tools and Capabilities”, with an inclusion date of 2026-08-14; GitHub API showed 950 stars on 2026-08-17 (the directory page snapshot had 874 stars, please refer to the repository page for the latest star count). The project homepage is agent-vision.anionex.me.

The positioning of both the directory page and the repository README is: giving pure text coding agents eyes. It does not provide “throw an image to a multimodal model and get a general description back”, but rather a set of visual toolkits plus a skill that teaches the agent when to call and how to accept results in sequence. The repository states that it has been verified in real Codex + DeepSeek sessions, and the same pipeline has also been end-to-end verified in Claude Code, Pi, Oh My Pi, and OpenCode.

The repository divides the content into two categories:
1. Visual Tool CLI + skill: Command-line tools such as glance, ground, detect, trace, crop, etc., plus the vision-tools skill. Any agent that can call the shell can use it.
2. Seamless access (optional upgrade): Local transparent proxy or single-file native extension. After installation, pasted images and the agent’s built-in image viewing tools can work directly without additional prompts.

Starting from 2026-08-13, the same maintainer launched a native DeepSeek Harness access package dsh-vision-toolkit (npm: @anionex/dsh-vision-toolkit). It uses a Git submodule chain in this repository to turn the above toolkits into a Bundle for Web / Headless Profile. There is also a separate entry in the community directory. This article focuses on the toolkit itself, with native dsh installation explained separately later.

Core Features¶

1. Image viewing with intent, not a vague generic description¶

The repository README directly points out the loss of common “stitching solutions”: most adapters only let the multimodal model generate a general description, which is then passed to the pure text model for assembly. An extra layer of semantics is added in between, and the details that are actually needed for the current step are easily lost.

The approach of agent-vision-toolkit is to first extract why the agent wants to look at this image. The source can be a user message, or the reason stated by the model when calling its built-in image viewing tool; this motivation is passed to the visual model as a focus hint. What is returned is a description tailored to the current task, not “please give a detailed description of this image”. The repository summarizes it as: visual capabilities do not have to be built into the model, they can also be built into the harness.

It is still a layer that converts images to text, and will not directly pass visual tokens to the pure text model. The overall quality is determined jointly by the main model and the multimodal model. This is listed as a limitation by the repository itself.

2. A set of composable CLIs¶

Tools are selected based on the problem, rather than a one-size-fits-all entry point:

glance only requires Python 3.11+. ground / detect / crop and long screenshot OCR use cases require pillow; trace requires pillow + numpy, and only requires vtracer when explicitly using the --outline outline fallback. The repository recommends installing optional dependencies in an isolated venv only for the tools that are actually used.

The coordinate convention uses original image pixels. The boxes output by ground / detect can be directly fed to crop or trace; if the same area needs to be inspected multiple times later, crop it to a file first and reuse it.

3. vision-tools skill and use case playbook¶

The CLI only solves “how to call it”. The skill solves “when to call it, in what order, and how to accept the final result”. The vision-tools included with the repository has a set of executable use cases, with documentation in skills/vision-tools/references/:
- Long screenshots / chat records / scrolling pages: Find low-content cut points, perform block-by-block OCR, retain speakers, timestamps and quotes, only merge truly repeated overlaps, and mark borders that need review. The repository gives a real-world Telegram running example.
- Restore UI from screenshots or design drafts: Give priority to reusing existing project components and materials, then combine native UI code, screenshots, rendering results and visual comparisons to align step by step.
- Restore icons, Logos, illustrations: Extract transparent PNGs; rebuild SVG when editable or lossless scaling is needed, and verify shapes, colors and transparent edges.
- Sketch / diagram / whiteboard → structured code: Identify nodes, text, connections and directions, output Mermaid, Graphviz, etc.
- Operate GUI based on screenshots: Locate controls, perform one operation, take a new screenshot and verify it, then proceed to the next step to avoid continuous clicking on expired screenshots.

The repository README also records several original effects to illustrate the granularity, not third-party evaluations: infographic screenshots restored to editable HTML/CSS; hand-drawn sketches restored to JupyterLab workspace (Codex + deepseek-v4-flash); quick UI restoration giving the first screenshot in about three minutes; using glance for multi-round image Q&A; using ground to locate screen elements, DeepSeek V4 playing chess autonomously; troubleshooting field name mismatches based on screenshots.

4. Optional seamless access layer¶

The CLI is suitable for “agents calling the shell themselves”. If you want pasted images and built-in image viewing tools to work directly, the repository provides optional access:

The proxy does not store API keys of upstream text models. The original Authorization from Codex / Claude Code is forwarded as-is; only the vision-side environment variables VISION_API_KEY, VISION_BASE_URL, VISION_MODEL are configured for the proxy. The full steps are in AGENT_INSTALL.md, and you are required to back up the host configuration before installation.

Installation and Activation¶

Commands given by the community directory¶

The original installation command on the plugin details page is:

dsh plugin add github:Anionex/agent-vision-toolkit

For reproducible installations, the directory page requires fixing the commit hash:

dsh plugin add github:Anionex/agent-vision-toolkit#<commit>

The directory page also reminds: the plugin runs with the permissions of the current dsh process, and may execute code during installation; you should check the source code repository and license before installation.

The boundary needs to be clarified first. At the time of writing, there is no package.json or dsh.bundle declaration in the root directory of this repository. The official DeepSeek Harness documentation states: packages without dsh.bundle can still be loaded into the profile via dsh plugin add, but will only be treated as ordinary dependencies, print a warning, and will not activate the configuration layer. Therefore, this directory command corresponds to the GitHub source code repository, and does not mean “install and you can paste images in the Web UI”. For native Bundle usage on dsh Web / Headless, the maintainer specifies using the @anionex/dsh-vision-toolkit mentioned below.

Repository Recommendation: Let the agent install according to the documentation¶

The easiest way is to send this sentence to the current agent (the original text comes from README_CN):

Follow the repository guidelines at https://github.com/Anionex/agent-vision-toolkit to install the visual toolkit and skill locally. If the vision API has not been configured, please find the configuration file according to the current system and guide me to fill in VISION_API_KEY, VISION_BASE_URL and VISION_MODEL.

If you also want to install the optional seamless access layer, send this revised version:

Fully read https://github.com/Anionex/agent-vision-toolkit/blob/main/AGENT_INSTALL.md, and install the appropriate visual proxy or native extension/plugin according to the agent application we are currently using. If the vision API has not been configured, please find the configuration file according to the current system and guide me to fill in VISION_API_KEY, VISION_BASE_URL and VISION_MODEL.

What you need to prepare is a multimodal API that supports OpenAI Chat Completions, OpenAI Responses or Anthropic Messages, along with its base URL, API key and model name.

Three-step manual installation¶

1. Point to a visual API

Write three environment variables into ~/.config/agent-vision-toolkit/env, and set the file permission to chmod 600:

VISION_API_KEY=sk-...
VISION_BASE_URL=https://openrouter.ai/api/v1
VISION_MODEL=google/gemini-3.6-flash

Any OpenAI-compatible endpoint that supports /chat/completions and image_url can be used. The repository gives examples: Alibaba Cloud Bailian https://dashscope.aliyuncs.com/compatible-mode/v1 + qwen-vl-max-latest. You can also set the following for the Python client / proxy:
- VISION_API_PROTOCOL=responses: Use /responses + input_image
- VISION_API_PROTOCOL=anthropic: Use Anthropic Messages; the Base URL should end with /v1 and not include /messages

Add LANG=en if you need English descriptions, default is Chinese.

2. Add the CLI to PATH

git clone https://github.com/Anionex/agent-vision-toolkit.git
export PATH="$PWD/agent-vision-toolkit/bin:$PATH"

To make it take effect permanently, add the export command to your shell configuration. If you also need the dsh subpackage in the repository, add --recurse-submodules when cloning, or run git submodule update --init --recursive in an existing checkout.

3. Install the skill

npx skills add Anionex/agent-vision-toolkit --skill vision-tools -a codex -g --copy -y

You can also copy skills/vision-tools/ to the current agent’s skills directory (for example ~/.codex/skills/), and it will take effect after restarting.

Use the native Bundle in DeepSeek Harness¶

If you only want to use it on dsh Web or Headless, and want pasted images, Settings and Artifacts to work, the command given by the maintainer is (npm version 0.1.19 at the time of writing):

dsh plugin --profile web add @anionex/dsh-vision-toolkit

For Headless Profile, replace --profile web with headless. This package declares dsh.bundle, pins peer dependencies to DeepSeek Harness ^0.1.0-rc.6, requires Node.js ^22.19.0 or >=24.0.0, and includes a pinned snapshot of agent-vision-toolkit that will not pull the upstream main branch in the background. After installation, restart the Web Profile, open Settings → Vision Toolkit, and you can use the built-in shared vision service for connectivity testing by default. Refer to the dsh-vision-toolkit repository for details; there is also a corresponding page in the community directory.

Typical Usage¶

The following commands and processes are from the repository README, not made-up cases.

1. Ask questions or perform OCR on a screenshot¶

glance screenshot.png -q "What is the main color of this image?"
glance screenshot.png --ocr

Do not expect a single glance command to process a long chat screenshot in its entirety. The built-in script in the skill will split the image into blocks, perform OCR block by block, merge overlaps and write out borders for review:

python3 skills/vision-tools/scripts/long_screenshot_ocr.py long-chat.png --mode chat -o long-chat.ocr.md

2. Locate controls, then crop or vectorize¶

ground screenshot.png "Send button"
crop screenshot.png --region 1563,514,1668,621 -o send-button.png
trace screenshot.png --region 1563,514,1668,621 -o icon.svg

ground analyzes a full image each time. If the target is small, add --region X1,Y1,X2,Y2 to only search within that box, and the output will still be original image coordinates. For dense pages that require a complete list, use detect to inventory block by area instead of expecting a single full-screen pass.

3. Restore pages or graphics¶

The order of the repository playbook can be summarized as: first locate and crop the required visual materials, then write code or rebuild SVG, then render, compare and verify. Quick UI restoration allows approximate colors and icon libraries, with the goal of quickly producing the first viewable screenshot; refined restoration continues to align using visual comparisons. GUI operations also follow “look once, act once, look again” to avoid continuous clicking on expired screenshots.

4. Paste images for direct use¶

After installing the optional proxy on Codex or Claude Code and restarting the host, you can directly paste images or let the model call the built-in image viewing tool. Pi, Oh My Pi, and OpenCode use single-file native extensions instead of the proxy, refer to the README in each directory for steps. The proxy connects to the upstream directly by default, and does not read Windows system proxies; if you need to explicitly use a local proxy, use --upstream-proxy or the environment variable VISION_UPSTREAM_PROXY.

Applicable Scenarios and Notes¶

It is suitable for:
- Main models are pure text models like DeepSeek, but you need to view error interfaces, design drafts and long screenshots daily
- What you need is not just “describe this image”, but also coordinates, cropping, OCR pipelines, UI / graphic restoration
- Already working in agents that can call the shell such as Codex, Claude Code, Pi, OpenCode, and want to reuse the same toolkit across hosts
- Need native paste and Web Settings in dsh, switch to the @anionex/dsh-vision-toolkit provided by the maintainer

Please note the following items, all from the directory page or repository documentation:
1. Permissions and License. When installed as a dsh plugin, the code runs with the permissions of the current dsh process, and the installation process may execute code. Check the source code and MIT license before installation. The community directory is not an official app store.
2. dsh is still a developer preview. The plugin interface may change. This repository itself is a Python toolkit; the native