Introduction

The philosophy of DSH is “everything is a plugin,” and the community directory is an independent site with no official affiliation with DeepSeek / Huanfang. For developers using DSH, if they wish for the agent to interact with real Windows desktop operations, they need to acquire three types of capabilities: reading window structure, executing input actions, and handling window states. The following introduces dsh-computer-use-win, which turns Windows computer use capabilities into a DSH plugin.

What is it

dsh-computer-use-win is a Windows computer use plugin maintained by Yu-tao-Li. The core consists of an MCP stdio server and a PowerShell/UIA desktop engine, providing 22 desktop tools externally. It solves the problem of how a DSH agent on Windows can view controls, view screenshots, execute mouse and keyboard operations, and manage windows.

Core Capabilities

Observation Capabilities

  • Provides three views of the UIA accessibility tree: control, content, raw.
  • Supports finding controls by name, automationId, class name, and value.
  • Screenshots use a three-tier chain: PrintWindow, WGC, with screen area as a fallback.
  • When OCR falls back, it returns word bounding boxes with screen coordinates; for CJK, it provides character bounding boxes.

Actions and Window Capabilities

  • Mouse operations.
  • Keyboard input.
  • UIA semantic actions.
  • Window management.

Backend and Security

  • Uses a persistent PowerShell backend with a JSON line protocol.
  • Built-in security mechanisms include: coordinate homing, emergency stop failsafe, identity guard, foreground verification, and Win key blacklist.

Installation and Activation

First, confirm the environment: Windows only is supported, and Node >=22 is required.

The installation command is as follows:

dsh plugin --profile web add github:Yu-tao-Li/dsh-computer-use-win

This command adds the plugin to the web profile. After installation is complete, restart dsh web, and the tools will appear in the form of mcp__wincu__windows_computer_use_*.

Note: Only one serverName: wincu line should be retained in the same profile.

Typical Usage

After the installation steps above, you can use the commands in the repository for local verification:

node mcp/server.mjs --self-test
node test/mcp-test.mjs
node test/notepad-e2e.mjs

These three commands are used for self-check, MCP testing, and Notepad E2E testing respectively.

Use Cases and Considerations

Suitable for developers building DSH workflows on Windows or agents that need to read desktop controls and execute input. The license is MIT.

Note the following limitations before use:

  • Windows only.
  • Requires Node >=22.
  • This project is a derivative work of cgissing/windows-computer-use (MIT), with upstream copyright notices retained in THIRD_PARTY.md.
  • WinUI, Chromium, and Electron drop PostMessage synthetic input; verified:false is an honest report, not a success.
  • UIA and synthetic input for elevation windows (UAC/admin programs) will be intercepted by Windows; they cannot be read or clicked.
  • Coordinates are based on physical pixels, using explicit Per-Monitor V2 DPI.
  • OCR provides character bounding boxes for CJK.
  • The plugin runs with the permissions of the current dsh process; you should check the source code and license before installing.

Links

GitHub:

https://github.com/Yu-tao-Li/dsh-computer-use-win

Directory page provided by the plugin clue:

https://www.skillhub.cn/plugins/Yu-tao-Li/dsh-computer-use-win