sage-guikit

DSH agents are text-native, yet a large share of practical tasks requires touching the Windows GUI: clicking a Save button, dragging a slider, typing into a form field, confirming a dialog has appeared. sage-guikit wraps the Win32 mouse and keyboard APIs, window management, and UI Automation into eleven directly callable model tools, with zero external runtimes and no resident daemon, so an agent can perceive the screen, act on it, and verify the result within a single session.

A three-layer precision stack covers the full interaction cycle. The structural layer (gui_uia) locates standard Win32, WPF, or WinForms controls by name or type, invokes buttons directly, and reads control text without any coordinate math. The visual layer (gui_screen for layout, gui_screenshot for the full desktop, gui_window_shot for a single window captured via PrintWindow even when occluded, at roughly one-thirtieth the file size of a full-screen grab) can overlay an annotated coordinate grid so the agent reads physical pixel positions straight from the image. The execution layer (gui_click, gui_drag, gui_type, gui_key, gui_scroll) drives the real pointer and keyboard. The verification layer (gui_wait) polls a pixel delta or window-title appearance to confirm an action took effect before the agent proceeds. Everything runs on the system-bundled PowerShell 7 and .NET assemblies; no Python, no API key, no service to manage.

Best suited to DSH agent developers and advanced Windows 10/11 users who need to automate local desktop applications and prefer a lightweight, session-scoped, plug-and-forget setup. The deliberate trade-off is clear: no vision model is involved; the agent reasons about coordinates itself through annotated screenshots or UIA queries, in exchange for a minimal runtime footprint and zero-configuration deployment.