computer-user-vision

The upstream computer-user plugin silently vanishes on DSH 0.1.2 and later. A single missing named import fails the entire ES module at load time: no tools, no settings card, no error message. Beyond that, its design predates visual models -- a screenshot returns a file path and asks the model to hand it to a local OCR plugin. For a model whose inputModalities include image, that is an unnecessary detour.

This fork commits screenshots through the host attachment service as a native image block so the model can look directly. Windows UI Automation enumerates the controls in the focused window and generates refs; computer_click accepts a ref or a name and lands on the exact rectangle without pixel arithmetic. Window rectangles come from the DWM visible frame, and PerMonitorV2 DPI awareness eliminates the systematic offset on scaled displays. Input tools accept an optional expect_window guard: if the foreground window does not match, the call is refused before any keystroke is sent. While operating, a pulsing gradient traces all four screen edges with a floating banner; Ctrl+Alt+Esc or the Stop button hard-stops every computer tool until the user re-approves.

Built for developers and automation authors running a visual model on DSH with Windows who need the AI to read the screen, click buttons, fill forms, and switch windows with precise coordinates, controllable focus, and an interruptible safety net.