2026-10-04

Introduction

DeepSeek Harness (DSH) uses a plugin-based architecture. Letting agents operate computers is a common requirement. dsh-cua provides this capability.

What It Is

This is a computer-control plugin backed by the cua-driver daemon (trycua). It wraps 14 model tools and uses DSH’s native registration, approval panel, and image attachment system, with no MCP dependency.

How It Works

One tool call = one spawn of the driver CLI (JSON over stdin, JSON over stdout); the actual state remains in the local daemon. Addressing priority: 1. Prefer element_token. Pixel coordinates are used for non-UIA regions such as canvas/video. Invalid parameter combinations are rejected directly at the plugin layer, and no input is emitted.

Features

  • cua_status: whether the driver daemon is online; includes startup guidance if offline
  • cua_list_windows / cua_list_apps: top-level window list (pid / window_id / title / bounds); running and installed application list
  • cua_window_state: accessibility element tree snapshot for a specified window: structured elements (element_token/role/label/frame) + Markdown tree + window screenshot
  • cua_desktop_screenshot: full-screen screenshot of the primary display (actual physical pixels)
  • cua_zoom: zoom into a local region of a window screenshot (small fonts / dense controls)
  • cua_verify_state: post-action assertion verification: element exists/enabled/selected/value equal, window bounds (px tolerance); results are three-state satisfied / unsatisfied / unknown, and unknown is never treated as success
  • cua_click / cua_type_text / cua_press_key / cua_hotkey / cua_scroll: click (left/right/middle, double-click / triple-click), type text, press key, hotkey, scroll—the target can be an element token, window pixels, or desktop coordinates
  • cua_set_value: set a control’s value directly through accessibility SetValue (more reliable than typing character by character, with read-back verification after writing)
  • cua_bring_to_front: bring a window to the foreground (usually unnecessary via background delivery paths)

Installation and Enablement

Prerequisites: a DSH deployment with a profile; the driver daemon is online.

dsh plugin --profile web add github:hfloveyy/dsh-cua

After restarting dsh web, verify:

dsh --profile web --dump-config

In the chat, ask the agent to “list all current windows and take a screenshot to see what is on the desktop.” A successful response means it is working.

Configuration

Override by id in the profile’s cordis.patch.yml:

- id: cua
  config:
    requireApproval: true   # 每个动作前弹审批面板（false 可关闭插件层审批）
    disabled: false         # true = 不注册任何工具
    timeoutMs: 15000        # 工具协作式超时（2000–120000）
    maxElements: 200        # 单次快照转发的元素上限（20–5000）
    maxRows: 120            # list_windows/list_apps 单次转发行数上限（10–1000）
    binDir: ""              # 自定义驱动二进制目录；留空按优先级解析

Security and Limitations

Security uses a two-layer gate. Plugin layer: the seven action-class tools return an ask decision through tools/pre-execute. Driver layer: the daemon maintains the permission mode (standard / bounded / unrestricted), session authorization leases, capability manifests, and the revoke command.

Known limitations:
- Platforms with a bundled plugin binary: Windows x64; macOS and Linux use the global driver (upstream install.sh).
- Multi-monitor support currently covers only the primary display (following upstream’s current version behavior).
- The driver’s browser CDP tool group (browser_navigate, browser_click, etc.) and application lifecycle group (launch_app, invoke_menu) are not yet registered.

Conclusion

dsh-cua provides element-level computer control for DSH. It is suitable for developers whose agents need to operate desktop applications. Please review the source code and license before installation.