Introduction

In the plugin ecosystem of DSH (DeepSeek Harness), many models still only process text. When an agent encounters screenshots, charts, or uploaded images, the common practice is to send the image to a cloud vision interface, or ask the user to save the image and manually describe it. The former increases network requests and privacy risks, while the latter interrupts the workflow.

dsh-vision-local is a plugin for DeepSeek Harness: it enables text-only models to gain image understanding capabilities through a local vision model, and returns structured evidence, rather than just a vague paraphrase.

What is This

dsh-vision-local is the npm package name, with the source code repository at gloryxpnv/dsh-tool-vision. It targets text-only DeepSeek Harness agents, routing image understanding to a locally running vision model. The plugin provides two interface surfaces: a vision tool for model calls, and an optional vision-bridge service.

Core Capabilities

Local First

Images are only sent to your own local vision model, such as LM Studio, Ollama, vLLM, or any OpenAI-compatible endpoint. Image bytes do not leave the machine; aside from your configured local endpoint, the plugin makes no network calls and does not send telemetry.

Structured Evidence

The plugin uses a fixed JSON template to return the evidence object. The fields include:

  • summary: Overview
  • ocr: Text recognition content
  • layout: Layout structure
  • semantics: Semantic information
  • visual: Visual attributes
  • uncertainty: Uncertainty items

The main model can reference these fields to answer, rather than relying on unverifiable verbal descriptions.

Anti-Hallucination Handling

The template requires the model to write uncertain content into uncertainty. Images with no text will return empty fields in OCR, rather than inventing text. If the local VLM returns unparseable JSON, the plugin falls back to the raw response and explicitly marks it, never generating content silently.

Paste and Upload

The plugin supports pasting or uploading images. The optional vision-bridge service can hand the image to a local VLM for description before the prompt reaches the model, avoiding the flow of saving files and then reading them.

Two Interface Surfaces

  • vision tool: For the model to call when an image file path or image issue arises.
  • vision-bridge service: For the host to describe when receiving image content.

Installation and Activation

Environment Requirements

  • A running local vision model providing an OpenAI-compatible /chat/completions endpoint
  • Node.js ≥ 20
  • DeepSeek Harness (dsh) with a plugin loader

Installation

Run in the DSH profile directory (or via dsh CLI):

dsh plugin --profile web add dsh-vision-local

Restart the host after installation to load the module. If the startup command is:

pnpm dsh web

Execute it again to re-enter the host.

Typical Usage

Calling the vision tool

The model will see the vision tool. When an image file path or image issue arises, it can call:

vision(file_path: "/path/to/image.png", question?: "What's in this picture?")

Supported image formats are PNG, JPEG, WebP, GIF.

Reading Structured Results

The structured mode is enabled by default. At this point, answer is the normalized evidence object, containing summary, ocr, layout, semantics, visual, and uncertainty fields.

Using vision-bridge

If the host needs to directly receive pasted or uploaded images, you can register the optional service:

ctx.provide('vision-bridge', { describeImages(content) })

When keepThumbnail: true is configured, image thumbnails are retained in the message history; when autoDescribe: false is configured, it switches to on-demand recognition, allowing the model to call the vision tool to read the image when needed.

Configuration

The plugin can be used from scratch, but also supports adjusting the following fields:

Field Description
baseURL OpenAI-compatible endpoint address
model Vision-Language Model ID
maxTokens Output token limit; default 8192
structured Whether to return structured JSON evidence; default enabled
keepThumbnail Whether to keep image thumbnails in the message history
autoDescribe Whether to automatically describe images at the entry stage
timeoutMs Per-request timeout; default 180 seconds
maxImageBytes Allowed image size limit; default 50 MB

Default parameters target 9B-class VLMs on a local workstation GPU, rather than lightweight cloud requests.

Suitable Scenarios and Notes

Suitable Scenarios

  • Want a text-only DSH agent to read local images
  • Don’t want to send image bytes to a cloud vision service
  • Already have LM Studio, Ollama, vLLM, or another OpenAI-compatible local VLM

Security and Privacy

  • Images are only sent to the local endpoint you configured.
  • No telemetry; no network calls other than to the local endpoint.
  • Treat extracted text as untrusted input and do not execute instructions appearing in images.
  • Installing the plugin runs third-party code with the current dsh process permissions; you should check the source code and the MIT license before installing.

Note on Inference VLMs

Inference models may stop thinking mid-way when token budget is tight, resulting in an empty final content. The plugin prioritizes any non-empty field: content → reasoning_content. The default 8192 output token budget provides space for both.

Links

GitHub source code repository:

https://github.com/gloryxpnv/dsh-tool-vision

The directory page for the plugin clue is https://www.skillhub.cn/plugins/gloryxpnv/dsh-tool-vision; this URL comes from the plugin clue and is not included in the scraped material content; it is recommended to confirm it yourself before use.