Preface¶

DeepSeek Harness (dsh) is an agent framework open-sourced by DeepSeek AI. The official repository summarizes its architecture in one sentence: Everything is a plugin. Model adapters, tools, sessions, and interfaces can all be replaced without modifying the framework source code. There is also an independent plugin directory site (deepseek-harness-plugin.com) in the community, used to search for repositories tagged with the dsh-plugin topic. It has no official affiliation with DeepSeek / Fangfu, so do not treat it as an official app store.

Pasting images in the web interface is a common operation: error screens, design drafts, tables, memes. The problem lies in the model capability declaration. Harness will decide whether to allow image attachments based on the current model’s inputModalities; the DeepSeek chat-completions endpoint only supports plain text, so attaching images when selecting this model will be rejected natively. The community already has visual plugins providing tools like view_image, which are suitable for file paths, but images directly attached in the GUI still cannot pass through for pure-text models.

dsh-vision-proxy fills this gap: image understanding is handed over to a visual model, while conversations are still answered by DeepSeek.

What it is¶

dsh-vision-proxy is an interface enhancement plugin maintained by Flyvhidbwo, licensed under MIT, and primarily written in JavaScript. At the time of writing this article, the GitHub repository version is 0.2.5, and it requires Node.js >=22.19.0 and DeepSeek Harness >=0.1.0-rc.6. It is categorized as “Interface Enhancement” on the community directory page; the star count is based on the repository, which showed 10 stars at the time of verification (the directory page showed 7 stars at that time, with a lag).

What it does can be summarized in one workflow:

User attaches image ──▶ deepseek-vision route ──▶ Translated by VLM (OCR + layout + details)
                   │                        │
                   ▼                        ▼
            DeepSeek answers ◀── Plain text conversation (images replaced with [image translation] text)

The plugin registers a new provider route deepseek-vision, wrapping the real DeepSeek adapter. Externally, it declares support for image input, so attachment previews will pass the check; before the actual request is sent, each attached image will first be converted to text via an OpenAI-compatible visual language model (VLM), then handed over to DeepSeek. The conversation brain remains DeepSeek, and image understanding is only an additional capability.

The repository README directly states the pain point: tool-based visual plugins solve the problem of “reading images by path”, but cannot solve the problem of “pasting images in the input box”. This plugin targets the latter scenario.

Core Features¶

Keep the brain, only add eyes¶

The default wrapped internal adapter is deepseek-official, displayed as DeepSeek + Auto Image Understanding in the model selector. Plain text messages will not be intercepted and will go directly to DeepSeek; only messages with image blocks will go through the translation process. The native read_image tool is also available under this route, because it reads the same model capability information.

Any OpenAI-compatible endpoint¶

The translation endpoint only needs to support the /chat/completions format. Common combinations listed in the repository are as follows:

The default main model is Bailian’s qwen3.7-flash. The API key reading order is: apiKey in configuration → environment variable $VISION_API_KEY → $DASHSCOPE_API_KEY. Non-anonymous entries without a key will be skipped instead of causing the entire chain to fail.

Each item in fallbackModels can carry its own baseURL / model / apiKey, and you can string multiple endpoints into a downgrade chain with one installation.

Fall back to local when no key is available, instead of hanging¶

autoLocalOllama is enabled by default. The plugin will probe http://localhost:11434 on startup, and if Ollama is found, it will automatically add it to the downgrade chain, so images will not leave the local machine. When there is no API key and no local Ollama, the translation will fail within a few seconds and prompt you to configure a key or install Ollama, instead of hanging silently.

The repository clearly states: No longer include any third-party anonymous free endpoints as the default fallback. The author explained that in actual tests, such endpoints (such as OVHcloud AI Endpoints) have strict rate limits and may hang without response. If you still want to use anonymous endpoints, you need to add them to fallbackModels yourself and set anonymous: true. Anonymous endpoints will enforce a 20-second timeout limit; immediately fail on HTTP 429 without waiting for Retry-After; failed endpoints will enter a 60-second cooldown period.

Ask during installation, mark endpoints during startup¶

The postinstall script will ask: Do you have a VLM API key? Answer y to go to the paid quick setup path, default N to go to the local / zero-configuration path. Non-interactive environments (CI, no TTY) will automatically skip this step, and the installation itself will not hang. At startup, a summary line will be printed (route id, wrapped provider, VLM model, endpoint, timeout, key source, etc., the key itself will not be printed), as well as a PRIVACY NOTICE indicating where the images are currently sent.

Caching and large image processing¶

Translation results are cached in-process based on the SHA-256 of the image bytes, with a limit of 200 entries, and will not be saved to disk. The same image will only be translated once in the current process, and can still hit the cache when reattached or switched to another conversation.

If the optional dependency sharp is installed, images exceeding maxImagePixels (default 4 million pixels) will be automatically scaled down before translation; if not installed, the original image will be sent directly. Dense UI screenshots may still lose small text, which is the capability limit of the visual model, not a plugin logic error. For scenarios with heavy OCR requirements, the repository recommends switching to a stronger model (such as qwen3-vl-plus) or increasing maxTokens (default 4096).

Installation and Activation¶

The installation command given on the community directory page, run in the DeepSeek Harness terminal:

dsh plugin add github:Flyvhidbwo/dsh-vision-proxy

For reproducible installation, fix the commit hash according to the directory page instructions:

dsh plugin add github:Flyvhidbwo/dsh-vision-proxy#<commit>

The repository README also provides the installation method for the web profile from npm (this plugin is mainly attached to the model selector of the web interface, so this path is generally used):

dsh plugin --profile web add dsh-vision-proxy

When accessing the official npm source slowly in mainland China, you can forward the mirror parameter to pnpm:

dsh plugin --profile web add dsh-vision-proxy --registry=https://registry.npmmirror.com

pnpm 10 and above block dependency build scripts by default. The first installation may exit with a non-zero code and prompt Ignored build scripts: dsh-vision-proxy, sharp. You need to approve both in the pnpm-workspace.yaml of that profile, then re-run the installation for the bundle to be registered successfully:

allowBuilds:
  dsh-vision-proxy: true
  sharp: true

If you encounter ERR_PNPM_MINIMUM_RELEASE_AGE_VIOLATION from pnpm 11 (new version released less than a day ago), the repository’s solution is: add minimumReleaseAge: 0 in the same file, or add --config.minimum-release-age=0 to the dsh plugin add command, then re-run.

The plugin runs with the permissions of the current dsh process, and may execute code during installation. Please check the source code repository and license before installing.

Typical Usage¶

1. Restart dsh web after installation is complete.
2. Select DeepSeek + Auto Image Understanding in the model selector (corresponding to route id deepseek-vision). This step is mandatory: only this route declares support for image input externally, so attachment previews will pass the check.
3. Paste the image into the conversation and add a question.

There is an example in the repository README: on the deepseek-vision route, using DeepSeek-V4-Flash as the brain, the user pastes a meme and asks “What do you see?”. The image is first converted to text with OCR and layout by the VLM, and DeepSeek then answers based on this text; the documentation states it is a single step, taking about 7.6 seconds. A default marker [图片转译] (translated as [Image Translated]) will be added before the translated text.

You can use the following command to confirm that there is only one plugin record in the configuration. Note: --dump-config will print the configuration in plain text, which may include keys.

dsh --profile web --dump-config | grep -A3 dsh-vision-proxy

The acceptance criteria according to the repository are:
- DeepSeek + Auto Image Understanding appears in the model selector.
- After pasting the image, first see [图片转译], then DeepSeek answers.
- When there is no API key and no local Ollama, the conversation should fail within a few seconds with guidance. This is the expected anti-hanging behavior, not a damaged installation.

How to Modify the Configuration¶

The bundle comes with default values, and most cases do not need modifications. To override, use id-based targeted override in $DSH_HOME/profiles/web/cordis.patch.yml, do not use insert:

- id: dsh-vision-proxy
  name: 'dsh-vision-proxy'
  config:
    baseURL: https://dashscope.aliyuncs.com/compatible-mode/v1
    apiKey: 'sk-…'          # Can also be left blank, will read environment variables instead
    model: qwen3.7-flash
    maxTokens: 4096
    timeoutMs: 120000
    maxImagePixels: 4000000
    marker: '[图片转译]'
    autoLocalOllama: true
    fallbackModels: []

The repository specifically reminds: In dsh’s patch semantics, insert appends to the list. Writing - insert: [{id: dsh-vision-proxy, …}] will cause both the built-in bundle entry and the user entry to be instantiated, and the deepseek-vision adapter will be registered twice, leading to undefined behavior. The top-level - id: will hit the existing line and replace the entire config; keys that are not written will fall back to the plugin schema’s default values, so you only need to write apiKey or model as well.

There is also a documented pitfall on Windows: File Explorer will cache environment variables, and $VISION_API_KEY exported after the process starts may not reach the running dsh, and the log will show skipped — no API key. The repository’s recommendation is to write the apiKey directly into the plugin configuration. In addition, dsh rc.6 does not load .env files, so you cannot bypass this by relying on it.

Applicable Scenarios and Notes¶

Applicable usage scenarios:
- Continue to use DeepSeek for coding, copywriting, and reasoning, but occasionally need to view screenshots, tables, or diagrams.
- Want to paste images directly in the GUI instead of saving them first and then calling view_image.
- Have Bailian / Zhipu AI keys in mainland China, or are already running an Ollama with visual capabilities locally.
- Need to string multiple VLMs into a downgrade chain instead of being tied to one.

Before using, please understand the boundaries:
- Images will leave the local machine unless baseURL points to a local service (such as Ollama). Images are sent as base64 via HTTPS to the configured VLM endpoint. Sensitive screenshots should use your own endpoint or local model; do not install this plugin if you cannot accept this.
- The plugin runs with the permissions of the current dsh process, can read workspace files, use existing credentials, and access the network. The tool approval box will not sandbox it.
- The community directory is not an official DeepSeek store. The installation command shall prevail from the directory page, and the source shall prevail from the GitHub repository.
- Pricing may change. The repository README provides the August 2026 reference price for Alibaba Cloud Bailian domestic station: a 1080p screenshot is estimated to be about 2000 tokens, and qwen3.7-flash costs about a few tenths of a cent; please refer to the real-time price on the console. Local Ollama does not incur this cost.
- DeepSeek Harness is still in developer preview, and the official repository states that incompatible changes may occur in the future. This plugin is based on the public interfaces of rc.6 (ctx.llm.registration, registerAdapter, proxy resolveModel / stream).

Summary¶

dsh-vision-proxy does not replace DeepSeek with a multimodal model, but builds a bridge at the GUI attachment layer: images are first converted to text with a [图片转译] marker, then enter the original plain text conversation. Use compatible endpoints such as Bailian if you have an API key, try local Ollama if you don’t; if neither is available, it will fail quickly instead of hanging a conversation round.

Directory page: https://deepseek-harness-plugin.com/zh-CN/plugins/dsh-vision-proxy/

GitHub: https://github.com/Flyvhidbwo/dsh-vision-proxy