Introduction

In the DSH plugin ecosystem, some sessions require processing local images: describing the scene, recognizing text, and continuing to answer based on image content. DeepSeek Harness allows integration with plugins, and dsh-vision-tools is a vision capability plugin prepared for this purpose. It passes local images to the OpenAI-compatible vision API for understanding, and then lets the current session continue processing.

Below is an introduction to its core capabilities, installation, configuration, and usage.

What is this?

dsh-vision-tools is maintained by moon09300731 and is licensed under MIT.

It provides the vision_understand global tool for DSH, enabling all sessions to call the OpenAI-compatible vision API to understand local images. The plugin defaults to using Zhipu’s GLM-4.6V-Flash and supports switching among four providers: zhipu, dashscope, siliconflow, and openai.

Core Capabilities

The capabilities listed below are derived from the plugin documentation:

  • vision_understand: Calls the OpenAI-compatible vision large model API to complete scene description, text recognition, and answering questions related to images.
  • Globally Available: vision_understand is registered as a global tool and is available to all sessions.
  • Three Entry Points: Paste screenshots, drag and drop images, or click a button to select files.
  • Automatic Saving to Disk: Paste, drag-and-drop, and button entry points will save images to $DSH_HOME/pasted-images/ and fill the input box with Please recognize this image: <path>.
  • Rate Limiting Fallback: When the main model is rate-limited, it can automatically fall back to VISION_FALLBACK_MODEL or a preset model of the provider for one retry.
  • Configuration Fallback: Supports global configuration via ~/.dsh/vision.env and workspace fallback configuration via .dsh-vision.env in the project directory; configuration is read in real-time, and no restart is required after changes.
  • Web Entry: Saves pasted images via the POST /api/vision-paste route and registers a “Recognize Image” button to the left of the DSH web input box.

Installation and Activation

First, execute the following plugin installation command:

dsh plugin --profile web add dsh-vision-tools

After installation, restart dsh web to activate the plugin.

Typical Usage

Below are common configurations and operation paths. First, create the global configuration file:

VISION_PROVIDER=zhipu
VISION_API_KEY=your_API_Key

If you need to use a specific model, API endpoint, or fallback model, you can add the following:

VISION_BASE_URL=https://open.bigmodel.cn/api/paas/v4/chat/completions
VISION_MODEL=glm-4.6v-flash
VISION_FALLBACK_MODEL=glm-4v-flash

The optional values for VISION_PROVIDER here include zhipu, dashscope, siliconflow, and openai. It defaults to Zhipu’s GLM-4.6V-Flash; if the main model is rate-limited, the plugin will attempt to fall back to VISION_FALLBACK_MODEL or a preset model of the provider for one retry.

After configuration, there are three ways to provide images:

  1. Use Cmd/Ctrl+V to paste screenshots from the clipboard.
  2. Drag images to the “Recognize Image” button on the left side of the input box.
  3. Click the “Recognize Image” button to select files.

After the above steps, the image will be saved to $DSH_HOME/pasted-images/, and the input box will be filled with Please recognize this image: <path>. After sending, the agent will automatically call vision_understand to recognize the image.

If you want the configuration to take effect only for the current project, you can place .dsh-vision.env in the project directory. This file uses the same format as ~/.dsh/vision.env and only applies to the current project.

Considerations and Notes

Suitable for quickly viewing screenshots, recognizing image text, or asking questions based on images in DSH web sessions. Before using, it is recommended to note these considerations:

  • Images will be sent out to external vision APIs via base64 encoding. If the images contain sensitive data, you can switch to local OCR, such as tesseract or paddleocr; or you can build a private network vision service and point VISION_BASE_URL to that service.
  • vision.env contains API keys and should not be committed to git. The documentation states that the repository’s .gitignore already ignores this file.
  • The paste route only listens to the DSH local port, and the image save path is $DSH_HOME/pasted-images/.
  • Rate limiting automatic fallback is only triggered by rate limit errors; business errors like invalid keys or parameter errors do not trigger a fallback and will result in an error report directly.
  • When the main model is the same as the fallback model, duplicate requests will not be made.
  • @deepseek-ai/dsh-tools is included with the DSH host runtime, and the plugin does not declare it as a dependency to avoid dsh-tools duplicate instances causing scheduler.prepare to crash.
  • The plugin runs with the permissions of the current dsh process. You should check the source code and license before installing. The license for this plugin is MIT.

Links

The relevant links provided in the plugin information are as follows:

  • Directory Page: https://www.skillhub.cn/plugins/moon09300731/dsh-vision-tools
  • GitHub: https://github.com/moon09300731/dsh-vision-tools

This allows you to integrate local image understanding capabilities into DSH sessions.