Introduction

In DSH Web, some Agents are connected exclusively to text models. When users upload images, Agents generate screenshots, browser screenshots, or reading image tools return images, the text models cannot directly perform image understanding; if image generation or editing is required, an additional image API must be integrated.

dsh-vision-mix is a vision enhancement plugin for DeepSeek Harness. It combines a text model, a vision model, and an optional image generation API into a Mix model, designed for image recognition, cross-turn follow-up questioning, image generation, and editing.

What is it

  • Plugin Name: dsh-vision-mix
  • Maintainer: haiziyao
  • License: MIT
  • Positioning: Vision enhancement plugin for DeepSeek Harness
  • Purpose: Integrating image recognition, image follow-up, image generation, and editing capabilities into the DSH conversation flow

Mix is the model entry point registered by the plugin; it is not a standalone model service that requires a separate API Key entry. Model credentials are still managed by DSH’s Provider configuration and credential system.

Core Features

  • Combine a text model, a vision model, and an optional image generation API into a Mix model
  • Enable Agents that originally do not support images to recognize uploaded images, understand web page screenshots, and continuously ask about image content
  • Support image generation or editing
  • Detect and declare model image capability in “Settings → Vision Mix”
  • Support testing and enabling image
  • Support force-enabling image
  • Support independent OpenAI-compatible image generation and editing API
  • Support vision_mix_image_generate to generate new images based on prompts
  • Support vision_mix_image_edit to edit user images or generated images that appeared in the current session
  • Support analyzing Agent screenshots, browser screenshots, and images returned by reading tools
  • Image recognition, basic conversation, intent judgment, and image generation can use different Providers respectively

Installation and Enablement

Installation

Execute in the DeepSeek Harness repository directory:

pnpm dsh plugin --profile web add dsh-vision-mix

After installation, start Web:

pnpm dsh web

Configure Base and Vision Models

First, configure the models:

  1. Open “Settings → Models”
  2. Configure base model
  3. Configure vision model
  4. Image recognition, basic conversation, intent judgment, and image generation can use different Providers respectively

If image generation and editing are needed, configure an independent OpenAI-compatible image generation and editing API Provider.

Detect and Declare image Capability

Complete this in “Model Image Capability” under “Settings → Vision Mix”:

  1. Select the configured vision model
  2. Select “Test and Enable image” or “Force Enable image”
  3. If “Test” is selected, the plugin will temporarily grant the model image capability
  4. Failed tests will automatically roll back
  5. After successful enablement, return to “Basic Settings”
  6. Select base model, image model, and optional intent recognition model
  7. Click “Save Route”

Enable Image Generation and Editing

After configuring the image generation Provider, test the image generation API in “Image Generation and Editing”.

Testing the image generation API will:
- Fixed use low
- Fixed use 1024×1024
- Fixed use PNG
- Incur one actual image generation fee
- Only preview and validate the result in memory
- Not permanently write test images to attachment storage

After enabling, you can call:
- vision_mix_image_generate: Generate new images based on prompts
- vision_mix_image_edit: Edit user images or generated images that appeared in the current session

Typical Usage

Create a new conversation and select Mix in the model selector. The following operations belong to typical scenarios supported by this plugin:

  1. Upload an image and let the Agent recognize the image content
  2. Continue asking about specific parts or details of the image in the next turn
  3. Request the Agent to take a screenshot and analyze the Agent screenshot, browser screenshot, or image returned by the reading tool
  4. Request image generation based on a prompt
  5. Request editing of user images or generated images that appeared in the current session

Credentials, Attachments, and Records

The plugin’s credentials and record boundaries are as follows:

  • The plugin does not contain or persist hardcoded API Keys
  • Model calls resolve credentials via the credential reference of the selected Provider
  • Configuration is written back to the original Provider’s model profile
  • API Keys are still stored in the DSH credential system
  • Session records save attachment id, not the host machine’s absolute path or image base64
  • vision_mix_attachment_query can only read attachments that appeared in the current session messages, tool results, or vision records
  • vision_mix_image_edit can only edit attachments that appeared in the current session messages, tool results, vision records, or generation records
  • Image preview validates both session id and call record id, and cannot enumerate across sessions

Default history paths:

Vision History:

$DSH_HOME/vision-mix/v1/calls/

Generation History:

$DSH_HOME/vision-mix/v1/generations/

Applicable Scenarios and Notes

Suitable for the following scenarios:

  • Text models are already configured in DSH, and you want Agents to support uploading images, screenshots, and cross-turn follow-up questioning
  • Vision models are already configured, and you need to detect and declare image capability
  • OpenAI-compatible image generation and editing API are configured, and you want to generate or edit images in the conversation

Notes before use:

  • The plugin runs with the permissions of the current dsh process
  • Check the source code and MIT license before installation
  • Testing image capability temporarily grants image capability; failures automatically roll back
  • Testing image generation API fixed uses low, 1024×1024, PNG, and will incur one actual image generation fee
  • dsh-better-sidebar is not a required dependency
  • Core routing, image recognition, cross-turn follow-up questioning, image generation, and conversation image display remain normal even if dsh-better-sidebar is not installed

Project Address

GitHub repository: https://github.com/haiziyao/dsh-vision-mix

Community directory can search by plugin name dsh-vision-mix.