Introduction

The DSH plugin ecosystem emphasizes “everything is a plugin”. If the main model is text-only, but the task requires viewing screenshots, recognizing text, or analyzing the interface, the image must be passed to an external multimodal API, and then the result retrieved as text.

Below is an introduction to AlloyPlane/dsh-eye-vision. It is a DSH plugin designed for DeepSeek Harness models that only support text input, capable of performing image understanding, OCR, and UI analysis via any OpenAI-compatible multimodal API.

Overview

The maintainer of the dsh-eye-vision repository is AlloyPlane, the license is MIT, and the upstream dsh-free-vision is also MIT.

Its working mechanism is: after sending the image path, the agent calls the image_understand tool, and then returns a text description. The main model does not require direct support for image input.

Core Features

The plugin provides the following verified capabilities:

  • Provides the image_understand tool
  • Supports arbitrary OpenAI-compatible multimodal API endpoints
  • Built-in free-tier providers for qwen, volcengine, and siliconflow
  • Supports large image multi-cropping
  • Supports allowed-directories whitelist (LUMA_ALLOWED_DIRS)
  • Settings take effect without requiring a restart after saving
  • Strips proxy environment variables from the engine subprocess

Installation

Add this plugin to DSH:

dsh plugin --profile web add dsh-eye-vision

After adding, the plugin provides the image_understand tool.

Typical Usage

The following example is used to pass image paths to the agent. image_source can be a local path, an http(s) URL, or a data URI.

See image: D:/path/to/screenshot.png
OCR: D:/path/to/document.png
UI analysis: D:/path/to/design.png (task_type: ui)

task_type supports the following values:

auto|general|ocr|ui|debug|describe

Configuration and Runtime Behavior

The settings file path is:

~/.dsh/free-vision.json

There are a few points to note during runtime:

  • API keys exist only in the settings file or environment variables, not written to the repository
  • Images are only sent to the configured endpoint
  • Proxy environment variables are deliberately stripped from the engine subprocess
  • Images support PNG, JPG, WebP, GIF, up to approximately 10 MB
  • If the engine needs to read images in the workspace, you can use the allowed-directories whitelist, with the corresponding configuration item being LUMA_ALLOWED_DIRS

Use Cases and Notes

Suitable for the following scenarios:

  • The main model cannot receive images directly, but needs to read screenshots, documents, or design images
  • Want to use existing OpenAI-compatible multimodal APIs rather than being locked into a single vendor
  • Need to convert image results into text descriptions, OCR text, or UI analysis within a DSH session

Please note before installation: the plugin runs with the permissions of the current dsh process; it is recommended to check the source code, license, and the API endpoint to be configured first.

If you plan to publish a fork, it is recommended to perform a GitGuard-style pre-push scan.

Related Links