Preface¶

In DSH’s plugin-based extension approach, the community directory is an independent site. The @dsh-extension/dsh-vision-bridge introduced in this article is a third-party plugin. It targets text-only DSH sessions and solves a specific problem: images may appear in sessions due to uploads or tool outputs, but text model requests are not suitable for carrying image blocks. If you send the long conversation history directly to a vision model to “see the image,” both cost and control become difficult to manage.

This plugin allows sessions to continue using the text model, only invoking vision_describe when image understanding is needed. Each vision call sends only the specified image and a focused question, treating the vision model as an on-demand external capability.

What Is This¶

@dsh-extension/dsh-vision-bridge is maintained by sfyyy and licensed under MIT. Its one-line positioning is:

On-demand vision for text-only DSH sessions: images become markers, and a vision_describe tool sends only image + question to an OpenAI-compatible vision model.

Simply put, it provides on-demand vision capabilities for text-only DSH sessions: images remain as images in the session and UI but are rewritten as text markers when entering the text model input layer. When the model needs to see an image, it calls the tool, and the OpenAI-compatible vision endpoint returns a text result.

Core Capabilities¶

On-Demand Vision Model Invocation¶

The plugin does not proactively send all images to the vision model. It registers a vision_describe tool, which the text model invokes only when needed. Each call sends only the image and question, without sending the long conversation history, thus keeping vision calls to the layer of “sending only what needs to be seen.”

Session Preserves Original Images, Rewrites Only Model Input¶

The plugin records image attachments appearing in the session via the agent/pre-step hook and builds an attachment index for vision_describe to resolve images by ID.

At the same time, it wraps session.deriveMessages() to ensure messages sent to the text model do not contain image blocks. Session logs and the UI still retain the original images; only the model input is rewritten.

Uses OpenAI-Compatible Vision Endpoint¶

The plugin supports any OpenAI-compatible /v1/chat/completions endpoint as the vision endpoint. In the DSH llm-pi-ai providers, it maintains only one vision-bridge provider route.

Restores Native Behavior When Disabled¶

Setting enabled: false shuts down the entire pipeline: no tool registration, no image rewriting, no admission bypass retention—restoring native behavior.

Installation and Enabling¶

Installation Command¶

Install from the npm registry, without using a local checkout:

dsh plugin --profile web add @dsh-extension/dsh-vision-bridge

If you always invoke the DSH CLI via npx, you can also use:

npx @deepseek-ai/dsh@0.1.0-rc.6 plugin --profile web add @dsh-extension/dsh-vision-bridge

--profile points to the profile you want to launch; web is the browser UI profile. If a new client bundle is added, you need to restart dsh web once for the UI to load the plugin.

Basic Configuration¶

You can configure it in DSH Web’s Settings -> Vision Bridge or edit ~/.dsh/vision-bridge.json:

{
  "enabled": true,
  "baseUrl": "https://api.openai.com/v1",
  "apiKey": "sk-xxxx",
  "apiKeyEnv": "",
  "model": "gpt-5.6-terra"
}

Configuration items include:

enabled: Whether to enable the entire pipeline.
baseUrl: OpenAI-compatible /v1/chat/completions vision endpoint.
apiKey: API key directly entered.
apiKeyEnv: Field for referencing an environment variable; mutually exclusive with apiKey.
model: Vision model name.

Directly entered keys will be synced to the DSH credential store and referenced as DSH_VISION_BRIDGE_API_KEY.

Configuration Priority¶

Priority from high to low:

Settings page (with schema defaults) -> environment variables -> config file

Available environment overrides include:

DSH_VISION_BRIDGE_BASE_URL
DSH_VISION_BRIDGE_API_KEY
DSH_VISION_BRIDGE_API_KEY_ENV
DSH_VISION_BRIDGE_MODEL
DSH_VISION_BRIDGE_ENABLED

Checking Runtime Status¶

You can request the settings endpoint provided by the plugin to view the current live value.admissionBypass and dependency-service status:

GET /_dsh/vision-bridge/settings

Typical Usage¶

Invoking vision_describe¶

vision_describe is used to let the vision model answer questions about images. It accepts:

attachmentIds: Image attachment IDs in the current session.
paths: Image file paths, resolved via DSH’s sandbox-aware file service, supporting png, jpeg, webp, and gif.
question: Required, must be a focused and specific question.

The total number of images per call is 1-4.

A parameter-level call can be written as a template:

vision_describe(
  attachmentIds: ["<current-conversation-attachment-id>"],
  paths: ["<image-path>"],
  question: "<question>"
)

Multiple Images¶

If you need to view multiple images at once, you can pass 1-4 images together to vision_describe and specify the content to observe or compare in the question.

Verification and Local Development¶

Run tests:

npm test

Tests cover marker rewriting, attachment resolution, event-log indexing, disabled shutdown, and text-only session behavior.

If developing from a local checkout, you can inject the local path:

dsh plugin inject /path/to/dsh-vision-bridge

Applicable Scenarios and Notes¶

Suitable for the following situations:

Occasionally needing to understand screenshots, uploaded images, or charts in text-only DSH sessions.
Wanting the vision model to receive only images and focused questions, not long conversation history.
Having an existing OpenAI-compatible /v1/chat/completions vision endpoint.
Wanting the UI and session logs to continue displaying original images while only rewriting the input sent to the text model.

Notes before use:

The plugin runs with the current dsh process permissions. It is recommended to check the source code and MIT license before installation.
apiKey and apiKeyEnv are mutually exclusive; do not rely on both to avoid ambiguity.
enabled: false disables tool registration, image rewriting, and admission bypass.
Attachment IDs must come from the current session; paths use DSH’s sandbox-aware file service.

Conclusion¶

The value of @dsh-extension/dsh-vision-bridge lies in equipping text-only DSH sessions with “on-demand image viewing” capabilities: images remain in the session and UI, the text model perceives images via text markers, and when truly needed, it invokes the vision model with minimal payload.

GitHub repository:

https://github.com/sfyyy/dsh-vision-bridge

The community directory page URL is not provided in the verified materials; you can use the npm registry installation command above during installation.

@dsh-extension/dsh-vision-bridge: On-Demand Visual Capability Integration for Text-Only DSH Sessions