dsh-mobile-gui-agent

While desktop automation has matured rapidly, mobile GUI automation still often suffers from fragmented toolchains, unstable input, and hard-to-verify actions. dsh-mobile-gui-agent extends DeepSeek Harness into the world of Android devices: it bridges to real phones or emulators via ADB, adds a dedicated mobile_gui_agent session to the Harness Web UI, and drives every task through an observe → decide → act → verify loop. No root access or persistent accessibility service is required—the plugin combines ADB screenshots with UIAutomator hierarchies to understand the screen and execute taps, swipes, text input, key events, and app launches, always returning to a fresh observation after each action to prevent drift.

The plugin’s core strengths are stability and safety. It ships with stale-element protection, adaptive screen stabilization, stuck detection, step and time limits, and support for wireless ADB and transparent Unicode input via an external helper. For consequential semantic actions—sending, publishing, deleting, purchasing, paying, transferring, calling, installing, or changing account security—the agent pauses and requests standard Harness approval before continuing. The model only interacts with the device through strict phone_observe and phone_act tools, never gaining arbitrary ADB shell access, which limits risk by design.

This makes the plugin ideal for developers and researchers who need to evaluate multimodal agents on real Android hardware, build mobile task benchmarks, or automate app testing. You can launch tasks from a blank session or follow the recommended flow of sending a short ordinary message first, then switching to the mobile_gui_agent tab to clearly separate conversation messages from phone commands. A fake device with scripted state transitions is included for keyless agent-loop tests, while real-device runs are best performed on a dedicated test phone with a non-production account.