DuoPlus CloudPhone Agent

Problem addressed

DuoPlus CloudPhone Agent targets stable UI automation on remote cloud phones: launching apps, locating page elements, clicking, typing, swiping, waiting, and verifying state. Like local device automation, it wraps common operations into repeatable commands, reducing the repetitive cost of writing raw ADB commands, assembling Base64 payloads, and manually parsing UI hierarchies.

How it works

The skill primarily sends JSON commands through scripts/send_command.sh, which the DuoPlus side executes. A typical flow includes:
- Element discovery: fetch the UI hierarchy, pull it locally, then use grep to find text, resource_id, or bounds="[x1,y1][x2,y2]".
- Clicking and input: prefer CLICK_ELEMENT by text; fall back to CLICK_COORDINATE when text matching fails. Before typing, tap the input field to focus it, then send INPUT_CONTENT.
- Page operations: supports LONG_COORDINATE, DOUBLE_TAP_COORDINATE, SLIDE_PAGE, GO_TO_HOME, and PAGE_BACK.
- Waiting and verification: many commands are fire-and-forget and do not return execution results, so take a screenshot after each action, compress it, and pull it locally for analysis; WAIT_FOR_SELECTOR can also wait for a target element to appear.

Boundaries and notes

• The coordinate system uses relative values from 0 to 1000, not pixels.
• Swipe coordinates should avoid overly regular integers, and start/end points should vary between consecutive actions to reduce the risk of looking like a fixed script.
• Text input depends on focus: run CLICK_ELEMENT or CLICK_COORDINATE before INPUT_CONTENT.
• Leave a local wait of 1 to 3 seconds between commands, and avoid using device-side shell sleep.