dsh-vimina: Integrating Vimina Capabilities into DSH Native Tools

DSH (DeepSeek Harness) follows the architectural principle of “everything is a plugin.” In Windows GUI automation scenarios, directly invoking low-level capabilities usually requires writing custom bridge scripts. This plugin registers the capabilities of Vimina (a .NET 8-based Windows GUI automation service) as native DSH tools, enabling models to directly invoke capabilities such as scanning, clicking, input, key presses, script control, screenshots, window management, and mouse operations, without relying on vision models.

Core Features¶

The plugin drives Vimina’s resident service through the stdio protocol (Vimina.exe serve) and maps low-level methods to DSH tools. There are 26 model-visible tools in total, covering the following areas:

• Window and scanning: list windows, scan foreground controls, scan by title, query controls by name or coordinates, and activate windows.
• Clicking and mouse: click by label, click by coordinates, scroll (with mouse wheel support), and get/move/drag/press/release the mouse.
• Keyboard operations: input text, key combinations, and press/release keys.
• Scripts and engine: execute/pause/resume/stop VMA scripts, query engine status, and retrieve script logs.
• Others: capability self-description and screenshots (with region capture support).

Installation and Mounting¶

It is recommended to install the plugin as a DSH profile dependency; the loader will automatically mount cordis.patch.yml.

dsh plugin --profile web add <包名或本地路径>

If manual mounting is required, modify dsh.profile.bundles or use the --patch parameter:

dsh web --patch path/to/dsh-vimina/cordis.yml

Usually, there is no need to manually configure the path; the plugin resolves the location of Vimina.exe automatically based on priority.

Configuration¶

The plugin supports the following configuration options (default values are shown in parentheses):

The automatic resolution priority for exePath is as follows:
1. Explicit configuration: configuration in DSH settings or the profile.
2. Environment variable: VIMINA_EXE.
3. Automatic detection: common installation locations (%LOCALAPPDATA%, %ProgramFiles%, etc.).
4. Fallback: Vimina.exe on the PATH.

Usage and Notes¶

Browser Usage Limitations¶

When operating Chromium-based browsers, the following rules must be followed:
- Real clicks are required: vimina_clickAt defaults to backend=false. Background clicks do not work on Chromium web pages; do not use backend:true.
- Scroll before scanning: Web page content must be scrolled first (vimina_scroll or vimina_key PageDown/End) before UIA can expose web page controls; otherwise, vimina_scan can only see the browser framework.
- getControlAt may hit containers: In web page scenarios, coordinate hits often return the outer container rather than the exact control, so the vimina_scan result should be treated as authoritative.

General Notes¶

• Scan before operating: vimina_clickLabel depends on the mapping from the most recent vimina_scan; rescan before operating.
• Sandbox limitations: The plugin lazily starts the Vimina process, which may be blocked with EPERM in restricted sandbox environments.
• No vision model required: vimina_scan and vimina_getElement return textual control names directly, without requiring OCR or a vision model.

Typical Usage Examples¶

1. Region screenshot
Capture the region at the top-left corner (100, 200) with width 400 and height 300, and save it as cover.png:

vimina_screenshot({ filename: 'cover.png', x: 100, y: 200, w: 400, h: 300 })

2. Page scrolling and scanning
Scroll the page and then scan controls:

vimina_scroll()
vimina_scan()

3. Control operations
Activate the window and click by label:

vimina_activate()
vimina_clickLabel('提交')

Summary¶

The dsh-vimina plugin standardizes Vimina’s GUI automation capabilities, providing DSH models with the ability to operate Windows directly without a vision model. Developers can integrate it into their workflows through simple configuration and commands, enabling more precise desktop automation.

GitHub repository