dsh-keymouse-use

Making an AI model genuinely "move" the Windows desktop has always hit two walls: input injected through virtual channels is easily flagged as non-hardware by applications, and DPI scaling makes the pixel coordinates a model sees disagree with what actually lands on the screen. dsh-keymouse-use closes that gap for DeepSeek Harness. A resident helper process reads JSON-line instructions over a stdin pipe and issues Win32 SendInput calls directly, producing keyboard and mouse events at the OS level that are indistinguishable from real hardware.

The entire pipeline is DPI-aware. Primary-screen capture, absolute mouse movement, and window queries all resolve to physical pixels, so the coordinate a model provides lands on exactly that pixel with zero intermediate scaling. Seventeen tools are exposed: the full key triad (press / release / tap), mouse press-hold for dragging, scroll-wheel, primary-screen PNG capture returning an absolute file path, window lookup by process name, tray activation with minimize-and-restore handling, minimize / restore / maximize, one-shot top-most pinning with a global release — plus a bundled runtime skill manual the model reads on its own via the skill tool, removing the need for hand-crafted prompts.

Best suited for developers running Windows desktop-automation loops inside DeepSeek Harness: multi-step logins, desktop application workflows, and the capture → observe → act iteration cycle. The plugin does exactly three things — input simulation, primary-screen screenshot, and window management — and ships with no image-recognition capability of its own. Whether a screenshot can actually be interpreted depends on the host environment providing vision or OCR tooling; the plugin binds to no specific visual solution. Windows single-screen desktop sessions only.