dsh-cua

Getting an agent to safely drive a local desktop GUI is harder than it looks. Raw pixel coordinates drift, there is no verification loop after an action, and focus-stealing wrecks the user workflow. dsh-cua wraps the cross-platform cua-driver (open-sourced by trycua) into 14 native DSH model tools so the agent targets controls through the accessibility element tree (UIA on Windows, Accessibility API on macOS). Expired snapshots fail with an explicit error instead of silently clicking the wrong spot. Every mutating action hits an approval panel by default while read-only observation passes silently. Screenshots enter the conversation as image attachments, so the agent reasons over the very same pixels used for targeting. A built-in assertion tool verifies element state after each action, and an unknown result is never treated as success. The entire stack uses DSH native tool registration with no MCP dependency, and the driver CLI is stateless and crash-isolated so it cannot drag the harness down.

Delivery defaults to background mode without stealing focus. Incomplete states like list truncation or snapshot degradation are honestly reported. The safety model is a two-layer serial gate: the plugin layer intercepts via the native approval panel, while the driver layer enforces permission modes, session authorization leases, and capability manifests as a backstop. Even if one layer is bypassed, the other remains in force, keeping the agent bounded by design.

Built for developers and teams using DSH to orchestrate agent workflows that require safe, auditable control over local desktop applications. If you want every GUI step to be approvable, verifiable, and traceable without introducing an extra protocol layer, dsh-cua delivers a cross-platform (macOS / Windows / Linux), ready-to-use solution out of the box.