dsh-screen-agent

AI agents can reason and plan, but they often lack a pair of hands to interact with the desktop. dsh-screen-agent fills that gap for DeepSeek Harness: screen capture, local OCR, mouse and keyboard control, and even driving the Windows Paint application to draw shapes, turning an agent from a talker into a doer.

The core capabilities span three areas. For screen recognition, it captures full-screen and multi-monitor screenshots and runs Windows built-in OCR to extract text with precise pixel coordinates, all locally with no network calls. For UI automation, it offers click, double-click, right-click, drag, safe Chinese text input via clipboard plus Ctrl+V, SendKeys-style key presses, and window activation, with support for secondary-monitor negative coordinates and DPI scaling. For drawing, it includes extensive battle-tested handling of Win11 Paint: mouse-acceleration compensation, curve and ellipse rendering, and pencil-based multi-segment assembly for small ellipses, so an agent can produce complete shapes such as a bowl or a duck.

The plugin is ideal for developers who want a local, Windows-based AI agent to handle screenshot analysis, form filling, cross-application workflows, or creative drawing experiments. Its zero third-party dependency footprint, built entirely on PowerShell and system APIs, keeps deployment overhead minimal.