Introduction

The following describes zfu691531-hash/dsh-gesture-mouse. This is a DeepSeek Harness (DSH) plugin that controls the macOS mouse using camera-based hand gestures. It processes visual recognition and system mouse events locally through the bundled Swift Helper, instead of sending every frame to a large model.

Core Features

The plugin primarily handles the following operations, all completed locally:
1. Index finger pointing: controls relative cursor movement.
2. Thumb and index finger pinch: completes a click or press-and-drag.
3. Index and middle fingers: controls scrolling.
4. Open/closed palm toggling: keeping the palm open can trigger a local panic (safety stop).
5. Safety mechanisms: each special gesture enforces a cooldown and deduplication, with a per-minute limit.

Installation and Configuration

Installation requires packing the plugin first, then installing it through the DSH CLI.

  1. Pack
npm pack
  1. Install the plugin
dsh plugin --profile web add ./dsh-gesture-mouse-0.1.0.tgz

After installation, restart DeepSeek Harness. By default autoStart: false, so it must be started manually.

  1. Configuration
    The default DSH bundle configuration is located in cordis.patch.yml. Key configuration items include:
    * autoStart: defaults to false.
    * sensitivity: relative cursor sensitivity.
    * cameraFPS: target capture frame rate, defaults to 30.
    * minimumConfidence: minimum confidence for Vision landmarks.
    * routes: named special gestures; each route must be explicitly set to enabled: true.

The safety floor cannot be weakened through configuration: the cooldown is at least 10 seconds, deduplication is at least 3 seconds, and the maximum trigger rate is 3 times per minute.

Usage Steps

  1. Start and authorize
    Call gesture_start, then approve Camera and Accessibility permissions when prompted by the system.

  2. Check status
    Call gesture_status to check cameraAuthorized, postEventAuthorized, FPS, and inference latency.

  3. Gesture control
    Extend the index finger and hold steady for about 200ms before moving the cursor; keeping the palm open can freeze it.

  4. Test
    Call gesture_test_trigger to test special gesture routing without opening a camera.

  5. Stop
    Call gesture_stop to release the camera.

Requirements and Permissions

  • System requirements: macOS 13 or later; DeepSeek Harness / dsh CLI; Node.js 20 or later.
  • Build requirements: Xcode / Swift 6 (required when building from source).
  • Permissions:
    • Camera: reads camera frames and processes them in memory only.
    • Accessibility / PostEvent: synthesizes mouse events through CoreGraphics.

The plugin does not request Screen Recording or Input Monitoring permissions.

Limitations and Notes

  • V0 baseline: parameters still require real-world calibration on real users and devices; it is not recommended to use it as the sole input device.
  • DeepSeek routing: in V0, it only generates a safety plan; it does not automatically start a model session or execute session.prompt.
  • Environmental factors: low light, occlusion, camera angle, and hand shape can affect Vision confidence.
  • Single-hand support: currently processes only one hand and does not support multi-hand coordination.
  • Lifecycle: the V1 lifecycle is tied to DSH; independent operation outside DSH is out of scope for now.

Conclusion

This plugin implements gesture control through a local Vision model and CoreGraphics, making it suitable as an auxiliary input component in the DSH ecosystem. For more details, refer to the GitHub repository.