DSH (DeepSeek Harness) uses a plugin architecture designed to enrich functionality through add-on components. The official DeepSeek interface is a text-only model. When an image is sent in the Web UI, the adapter throws an UNSUPPORTED_CONTENT error because the message contains an ImageBlock, causing the session to be interrupted. The multimodal vision bridge plugin proxies llm/stream requests, converts images into structured Chinese descriptions, and registers active vision tools, enabling text-only models to process images.

Plugin Overview

dreamrift/dsh-vision-bridge is a multimodal vision bridge plugin in the DSH ecosystem, maintained by developer DreamRift. It is categorized as “Model Inference” and is licensed under the MIT License.

The core value of the plugin is “bridging”: it does not modify DSH core source code. Instead, through interception and proxying mechanisms, it provides vision capabilities to models that do not support images (such as dsh-llm-deepseek), while remaining compatible with natively multimodal model pipelines.

Core Features

The plugin mainly provides the following four core capabilities:

  1. llm/stream request interception and proxying
    The plugin listens to and intercepts llm/stream requests. When a message contains an ImageBlock, the plugin converts it into a structured Chinese description (including transcribed text from the image), replaces it with a text block, and resends it. This process supports cross-turn caching to maintain KV cache stability and includes a fallback mechanism on failure.

  2. Active vision tools
    The plugin registers two tools for the model to call:

    • view_image: view a local image and answer questions about it.
    • ocr_image: transcribe an image verbatim.

  3. Native multimodal routing detection
    At runtime, the plugin detects the currently used model route. If it detects a model that natively supports images, such as DSH official llm-pi-ai (inputModalities includes image), the request is passed through directly to avoid vision bridge overhead. Vision proxying is enabled only for text-only routes (such as deepseek-official).

  4. Settings page hot updates
    The plugin supports modifying the vision model endpoint, model name, and OCR model in the DSH settings page (plugin configuration). Configuration changes take effect without a restart.

Installation and Enablement

Official Installation

It is recommended to use the official installation command; the plugin is listed in the directory:

dsh plugin --profile web add "github:DreamRift/dsh-vision-bridge"

Manual Installation

If you need to install from source:

  1. Clone the repository and generate the package:
    cd dsh-vision-bridge
    npm pack
  1. Declare the dependency in $DSH_HOME/profiles/web/package.json:
    {
      "dsh": { "profile": { "bundles": ["dsh-vision-bridge"] } },
      "dependencies": {
        "dsh-vision-bridge": "file:<本仓库绝对路径>/dsh-vision-bridge-0.4.0.tgz"
      }
    }
  1. Install the profile dependencies:
    cd "$DSH_HOME/profiles/web"
    pnpm install

Configure the API Key

Write the vision model API key in the DSH root configuration file $DSH_HOME/.credentials.yaml:

QWEN_MM_VISION_API_KEY: sk-<your-key>

After installation, restart dsh web. The log should show the [vision-bridge] 已启用 message.

Typical Usage

After installation and configuration, in a DSH conversation:

  1. Automatic image processing: Upload an image directly. The plugin automatically converts it into a text description and inserts it into the context, allowing the model to answer based on the image content.
  2. Using tools: In a prompt or when actively called by the model, you can use view_image to view a local file, or ocr_image to obtain the text content in an image.
  3. Changing configuration: Go to Settings page → Plugin configuration, modify the vision model endpoint or OCR model name, and the changes take effect immediately after saving.

Notes

  • Runtime environment: Supports Windows, Linux, and macOS. No Python or WSL environment is required.
  • Dependency management: The plugin has no hardcoded npm dependencies. If runtime dynamic imports are missing packages such as dsh-settings/schemastery/dsh-tools, the corresponding features degrade automatically.
  • Core modifications: The plugin does not modify DSH core source code and does not write session events.
  • Script modifications: If you modify the route admission exemption script after deployment, the DSH backend must be restarted for the changes to take effect.
  • Ecosystem positioning: This plugin is a community project and has no affiliation with DeepSeek official.

Summary

dsh-vision-bridge solves the problem that text-only models in the DSH ecosystem cannot process images by using lightweight proxying and tool registration mechanisms. It optimizes performance by leveraging native route detection and provides flexible configuration through the settings page. It is suitable for developers who need to enhance model vision capabilities in a local DSH environment.