Introduction

When developing agents, you often encounter issues like: the model is purely text-based, so when a user pastes a screenshot and asks, “What is this error?”, the model can’t answer; to generate an image, you have to open a separate tool, download the file, and drag it back into the session; or to operate a webpage, mainstream solutions either run an invisible headless browser or directly hijack the user’s own browser profile, which carries significant risk.

dsh-mindseye (MindsEye) targets these three gaps. It is a plugin for DeepSeek Harness (DSH). In a nutshell: Intent-driven vision, image generation, and visible browser automation for DeepSeek Harness. The following sections introduce it in order of features, installation, configuration, and considerations.

What is this

MindsEye is maintained by kanchengw, with the repository at kanchengw/dsh-mindseye. The current version is 0.2.8, licensed under MIT, and requires a Node engine of >=22.19.

Its design principle is to keep the DSH session as the primary interaction interface: images are retained as DSH attachments, and generation results are displayed in the session as native DSH attachments, so users don’t need to manually select local files or shuttle between multiple tools.

Core Features

Image Understanding

  • Retain DSH image attachments; image turns automatically mount the vision tool, while text-only turns keep a single active entry point.
  • mindseye_read_image: Answers single/multi-image questions, supporting focused visual tasks such as OCR, layout, charts, colors, and pixel differences. Returns a structured result containing the image, evidence, answer, and invocation metadata.
  • mindseye_ground: Locates the target and returns pixel bounding boxes for downstream actions like clicking or cropping.
  • When a text-only turn requires vision capabilities, mount the vision tool manually via mindseye_vision_activate.
  • mindseye_plan is responsible for extracting the current request and preparing the intent context for downstream tool usage.

Image Generation and Editing

  • mindseye_generate_image: Sends the user’s image request to the configured image generation route, and returns the generation result as a native DSH attachment displayed in the session.
  • mindseye_edit_image: Sends the DSH image attachment and the edit request to the configured image editing route.
  • Note: Generated images are not automatically saved to the project, and validation passes are not run; you need to handle disk persistence yourself.

Browser Automation

Once gui.enabled is configured, the plugin opens a separate visible Chrome or Edge session. The GUI tools can open pages, snapshot, wait, click, type, press keys, scroll, and close the session, corresponding to mindseye_gui_open / snapshot / wait, mindseye_gui_click / type / keypress / scroll, mindseye_gui_close.

When encountering CAPTCHAs, logins, or permission confirmations, the run pauses via a native DSH question card, and the user can:

  1. Take over the visible browser to complete the step manually;
  2. Skip the first handover prompt if the step is likely already completed;
  3. Abandon the current run.

After the user resumes, MindsEye first checks the page state before returning control to the model. Additionally, a new snapshot is required after every GUI action to prevent element references and coordinates from silently expiring.

Optional Memory Tools

The plugin also provides a set of optional memory tools that expose explicit DSH operations for storing, retrieving, searching, and comparing image-related records.

Installation and Enablement

Installation is a two-step process:

npm install dsh-mindseye
npx @deepseek-ai/dsh plugin --profile web add dsh-mindseye

After installation, restart DSH Web, then configure at least one vision route in the MindsEye settings card. Unconfigured focused vision routes (such as extract or locate) will fall back to the general understanding route when available, so the minimum viable configuration only requires one route.

Configuration

Settings are made via the DSH settings card or plugin configuration. The main configuration items are as follows:

  • vision.routes: Three independent routes: understand / extract / locate.
  • vision.fallbacks: Fallback routes for vision calls.
  • image.generate: Image generation route, attempted in order.
  • image.edit: Image editing route, attempted in order.
  • gui.enabled: Enable visible browser tools, default is off.
  • gui.browser: auto, chrome, or edge.
  • gui.restrictHosts: When set to true, enables a host whitelist.
  • gui.allowedHosts: A list of hosts allowed to be accessed when host restrictions are enabled.
  • gui.maxSteps and gui.timeoutMs: Step count and timeout upper limits for a single browser run.

Vision routes use OpenAI-compatible Chat Completions or Responses API; image routes support JSON and multipart request bodies, allowing different image providers to be configured independently.

Data and Security

Several design points worth confirming before integration:

  1. Image bytes and questions are only sent to the corresponding provider during the MindsEye tool call; credentials come from DSH credentials, environment variables, or plugin settings, and are only sent to the matching provider.
  2. Browser automation only starts local Chrome/Edge subprocesses when explicitly enabled, using an isolated session; it does not attach to the user’s existing browser profile or execute downloaded code; navigation can be restricted to the whitelist via gui.restrictHosts / gui.allowedHosts.
  3. Image capability models retain native DSH image blocks; the text-only fallback path uses isolated temporary files created for the current paste operation.

Suitable Scenarios and Considerations

Suitable scenarios: Pure text models need to read screenshots, charts, or perform OCR; needing to generate and edit images directly within the session; needing the model to operate webpages while requiring the process to be visible and manually takeoverable.

Considerations:

  1. The plugin runs with the permissions of the current dsh process; it is recommended to check the source code and license before installing. This project is under the MIT license, and the repository is publicly verifiable.
  2. gui.enabled is off by default; browser automation only starts when explicitly enabled, so no local browser subprocess will run if it is not turned on.
  3. The visual/image services needing integration must be compatible with the OpenAI-compatible API or support JSON/multipart request bodies; specific routes are configured one by one in the settings card.

If you wish to participate in development, the repository provides standard commands:

pnpm install
pnpm test
pnpm typecheck
pnpm build

Summary

dsh-mindseye integrates image understanding, image generation, and visible browser automation into the DSH plugin ecosystem, while keeping interaction, permission boundaries, and manual takeover within the DSH session. If your DSH workflow lacks multimodal and web operation capabilities, give it a try.

Repository address: https://github.com/kanchengw/dsh-mindseye

Community directory listing address: https://www.skillhub.cn/plugins/kanchengw/dsh-mindseye (This address is from a plugin listing lead, please refer to the actual site). DSH’s philosophy is “everything is a plugin”; the aforementioned community directory is an independent site and has no official affiliation with DeepSeek or Fangfang.