Introduction

DeepSeek Harness (DSH) uses a plugin-based architecture to decouple core functionality. The dsh-keymouse-use plugin uses a persistent helper process (bin/helper-x64.exe) to simulate real keyboard and mouse input through the Win32 SendInput API. It also provides screenshot and window management capabilities, helping models complete tasks on Windows, from simple key presses to complex window operation workflows.

Core Capabilities

The plugin mainly includes the following functional modules:

  1. Keyboard simulation

    • key_press(key): Press a key, used for key combinations.
    • key_up(key): Release a key.
    • key_tap(key): Press and immediately release, used for a single key press.

  2. Mouse simulation

    • mouse_press(button): Press a mouse button.
    • mouse_up(button): Release a mouse button.
    • mouse_move(x, y): Move the mouse pointer (physical coordinates).
    • mouse_roll(up, amount?): Scroll the mouse wheel.

  3. Screenshot

    • screenshot(): Capture the primary screen, returning the absolute PNG path, width, height, and the hwnd/title of the foreground window at that time.
    • screenshot_window(hwnd?): Capture a specified window (default is the foreground window), returning the path, dimensions, capture mode, and window state.

  4. Window management

    • win_find(process): Find a window by process name.
    • win_activate(hwnd): Activate a window, handling minimized or tray states.
    • win_minimize / win_restore / win_maximize: Minimize, restore, or maximize a window.
    • win_top(hwnd, on): Set a window as topmost; use together with release_top.
    • get_window(): Get detailed information about the current foreground window.

  5. Cleanup

    • release_all(): Release all held keyboard and mouse buttons with one action, preventing stuck keys or buttons.

Installation and Activation

Local directory installation (development or intranet environments):

dsh plugin --profile <name> add <path-to-dsh-keymouse>

Install via npm (after publication):

dsh plugin --profile <name> add @temsmie/dsh-tool-keymouse

When installing in the desktop client (DeepSeek Harness client 0.2.0), you need to operate through the plugin management panel. After installation, the desktop client must be restarted for the tools to be registered.

Typical Usage

Below are examples of common operation sequences:

1. Activate a window and perform actions
First find the window handle, then activate the window, and then perform the operation.

// win_find('QQ') → win_activate(hwnd) 唤起窗口（不点图标）
const hwnd = win_find('QQ');
win_activate(hwnd);
key_tap('enter'); // 在窗口中输入回车

2. Use key combinations
First press the combo key, then press the target key, and finally release all keys.

// key_press('ctrl') → key_tap('s') → release_all
key_press('ctrl');
key_tap('s');
release_all();

3. Mouse click and movement
Press the left mouse button, move to the specified coordinates, then release.

// mouse_press('left') → mouse_move(...) → release_all
mouse_press('left');
mouse_move(800, 600);
release_all();

4. Screenshot and observation
First capture the screen, have the host environment view the image, confirm the location, and then perform the operation.

// screenshot() →（环境支持看图时）查看图像 → 再定位操作
const path = screenshot();
// 假设宿主环境支持读取 path 并理解图像内容
// 然后根据图像内容计算坐标并执行鼠标移动

Notes

  1. Platform limitation: Only Windows systems are supported.
  2. DPI awareness: The helper process is set as DPI-aware. All coordinates and dimensions are based on physical pixels. The coordinates provided by the model are the same coordinates seen in the screenshot, so no scaling conversion is required.
  3. No visual recognition: The plugin itself does not include image recognition. screenshot returns a PNG file path, and whether the image content can be understood depends on the host environment (whether the model supports image input or whether other vision plugins are installed).
  4. Configuration scope: DSH loads a copy in the profile directory, not the source directory. If you modify the source, you must reinstall or publish a release for the changes to take effect.
  5. Permission requirements: The plugin runs with the permissions of the current DSH process. Ensure that the process has permission to operate on the target window.

Ecosystem Context

The core philosophy of DSH is “everything is a plugin.” Community directories (such as skillhub.cn) are independent sites and have no official affiliation with DeepSeek or High-Flyer AI. This plugin is published through community directories or GitHub for developers to use as needed.

  • Directory page: https://www.skillhub.cn/plugins/msn233/dsh-keymouse-use
  • GitHub: https://github.com/msn233/dsh-keymouse-use