dsh-computer-use

Most DeepSeek Harness agents are confined to text and terminal interactions. The moment a task requires looking at a screen, clicking a button, or bringing a window to the foreground, teams reach for brittle clipboard-screenshot scripts or patch the host source tree. dsh-computer-use injects full desktop control into the agent toolbox as a standalone Cordis bundle: screenshot observation returns a model-visible image attachment bound to a SHA-256 hash and a short-lived screenshot ID; coordinate actions map proportional image pixels back to physical desktop coordinates; and the native accessibility tree (Windows UIA, macOS AX, Linux AT-SPI) lets the model locate elements by name, role, or Automation ID and execute semantic actions such as invoke, focus, toggle, and expand. No Harness source modification or agent-preset edits are required — the bundle is live the moment it is installed.

The plugin deliberately links observation to action through evidence: every successful coordinate or semantic action consumes all of the agent's prior screenshots, semantic snapshots, and window-list records, forcing the next step to be grounded in freshly captured desktop state. Window focus is a single native call that raises the target and captures by its real-time bounds in one shot, sidestepping the classic Windows console-window foreground-stealing problem. The macOS and Linux backends wire into their respective system interfaces but are explicitly labelled reference implementations; production workloads should target Windows.

Ideal for developers and power users driving multi-step desktop workflows in Harness — form filling, local-application interaction, cross-window data reconciliation. If your agent needs to see before it acts and every step should leave a model-readable evidence trail, this bundle offers the most complete desktop control surface currently available in the Harness ecosystem.