Preface

In the DSH Web session or agent flow, if the currently selected model is a text model such as DeepSeek V4 Flash or DeepSeek V4 Pro, but the request contains images, the model itself cannot directly recognize these images. dsh-vision solves this problem: when a request contains images and the target model cannot recognize them, it first describes the images, then replaces the image blocks in the provider call with the text description. Session logs and the UI still retain the original images.

What is this

yepyeel/dsh-vision is a DSH plugin designed to provide visual recognition capabilities to text models. Its core behaviors are:

  • When a request contains images and the target model cannot recognize them, describe the images.
  • Only replace the image blocks in the provider call with text descriptions.
  • Session logs and the UI retain the original images.
  • License is MIT.

Core Features

Image Description and Replacement

When a request contains images and the target model cannot recognize them, the plugin describes the images and replaces the image blocks in the provider call with text descriptions.

The replacement only happens at the call layer sent to the provider. Session logs and the UI still display the original images.

Visual Model Selection Strategy

The plugin has two modes:

  1. Auto mode
    Iterate through added models that declare image input in the order of provider registration, using the first successful one; if unavailable, fall back to system OCR.

  2. Specified model mode
    Only use one visual model selected in Settings. Failures do not retry, nor do they fall back to other models or system OCR.

System OCR Fallback

System OCR fallback depends on the platform:

  • macOS: Vision framework
  • Windows: Windows.Media.Ocr
  • Other platforms: Tesseract can be installed and used in PATH

Note: System OCR is text extraction, not scene description.

Description Result Caching

The plugin caches description results based on “Attachment + Model” or “Attachment + OCR”. The cache persists until the process ends. This means within the same process, subsequent turns will not initiate the same visual model call or OCR call again.

Capability Declaration and Request Rewriting

At startup, the plugin wraps ctx.llm.resolveModelInfo to declare image input capabilities for text models, enabling the image-admission gate and the read_image tool.

Subsequently, the plugin rewrites the request in the llm/stream waterfall: replacing images with descriptions and invoking the visual model.

Installation and Enablement

Prerequisites

The following conditions must be met before installation:

  • DSH is already installed.
  • The dsh CLI is in PATH.
  • If using visual model description: At least one added model declares image in inputModalities.
  • If using OCR fallback:
  • macOS uses Vision framework, no extra installation needed.
  • Windows uses Windows.Media.Ocr, no extra installation needed.
  • Other platforms require Tesseract to be installed in PATH.

Installation

First enter the plugin directory, then run:

dsh plugin --profile web add .

After installation, restart dsh web to load the new bundle layer and client settings page.

Removal

If you need to remove the plugin, run:

dsh plugin --profile web remove dsh-vision

Typical Usage

Below is a reproducible basic flow:

  1. Confirm that at least one model is added that declares image in inputModalities, or confirm that the current platform supports system OCR.
  2. In the plugin directory, run:
   dsh plugin --profile web add .
  1. Restart dsh web.
  2. Open Settings → Visual Recognition, select Auto or Specify Vision Model.
  3. Use a text model in a session containing images. When the request is sent to the provider, the plugin replaces the image block with the described text; Session logs and the UI still retain the original images.

The same configuration values are saved in:

$DSH_HOME/settings.yaml

Where located in:

dsh-vision:

Applicable Scenarios and Notes

Suitable scenarios:

  • You are using text models such as DeepSeek V4 Flash, DeepSeek V4 Pro, etc.
  • Image attachments appear in the session.
  • You want the text model to continue processing the context, while image descriptions are provided by other visual models or system OCR.
  • You have added a visual model that declares image input, or the current platform supports system OCR.

Notes before use:

  • Only models declaring image in inputModalities are selected.
  • Gateway models that can recognize images but are not declared will not be selected.
  • Specified model mode is fail-closed: missing, text models, or failed models will not be replaced.
  • System OCR is text extraction, not scene description.
  • Settings RPCs only loopback.
  • Since image capability is declared, sessions containing images can switch back to text models; historical images will be described in the next llm/stream pass.
  • The plugin runs with the current dsh process permissions. Check the source code, dependencies, and license before installing. The license for this plugin is MIT.

Links

  • GitHub: https://github.com/yepyeel/dsh-vision
  • Plugin Directory Page: https://www.skillhub.cn/plugins/yepyeel/dsh-vision