Remote Browser Automation and Visualization Control Tool

The Core Problem: Verification Blocks in Headless Automation

When AI agents automate web tasks (e.g., posting content, managing dashboards), they often rely on headless browsers. While effective for simple API-like interactions, this approach fails when encountering CAPTCHA challenges, slider verifications, QR code logins, or risk-control pop-ups. The AI cannot perceive or interact with these human-required steps, causing automation to halt. Modern platforms' anti-automation mechanisms make this a frequent and critical bottleneck.

The Solution: Visual Control and Human-in-the-Loop Collaboration

This skill addresses the issue by deploying a fully visible and interactive browser environment, enabling seamless handoff between AI and human control:

1. Visual Deployment: On a Linux server, it launches a Chromium browser on a virtual display (e.g., DISPLAY=:99) and streams its video feed via noVNC (powered by websockify) to a web page on port 6080. Users can view and interact with the remote desktop directly from their browser.
2. AI Control Interface: The browser starts with remote debugging enabled (--remote-debugging-port=9223). The AI agent connects to 127.0.0.1:9223 using the standard Chrome DevTools Protocol (CDP), allowing it to perform actions like navigating, clicking, typing, reading DOM, and taking screenshots.
3. Collaborative Workflow: When the AI encounters an insurmountable verification step, it pauses. The user then completes the action (e.g., login, captcha) manually in the noVNC interface. Once done, the user signals the AI to resume, which then takes over control via CDP to finish the task.

Key Principles and Steps:
- Post-deployment, all browser operations must be performed through the skill's CDP endpoint (9223). Using the agent's built-in headless browser or spinning up new instances is prohibited to ensure user and agent see the same screen.
- For security, noVNC is password-protected by default, and the CDP port (9223) should not be exposed to the public internet.
- The services are configured as a system service (systemd) for automatic recovery on server reboot.

Use Cases and Explicit Boundaries

Ideal Scenarios:
- Web Automation Requiring Manual Verification: Automating posts on platforms like TikTok/Kuaishou, managing Weibo accounts, or logging into various CMS backends.
- Collaborative Debugging: Any situation where a user needs to watch and potentially intervene in AI-driven browser operations in real-time.

Not Suitable For:
- Pure API-Based Tasks: If a website provides a comprehensive API, it should be preferred over browser automation.
- High-Security Environments: Not recommended for production environments where any form of remote desktop exposure is unacceptable.
- Linux Servers Without root/sudo: The Linux mode requires elevated privileges to install system components. The Windows mode has no such restriction, as it directly takes over the user's local Edge/Chrome browser.