BrowserSkill

Problem Context

Browser automation tasks often require accessing pages where the user is already logged in, such as form filling, UI testing, or data scraping. However, directly controlling the browser window can disrupt normal browsing or expose sensitive information. Traditional approaches run in isolated environments (e.g., headless browsers), losing session context and requiring re-login, which reduces efficiency.

Core Working Principle

BrowserSkill drives the user's real Chromium browser via the bsk CLI and extension, executing automation in an isolated Agent Window while the user's normal windows remain independent. The core workflow includes:
- Session Management: Use bsk session start to create a session and open the Agent Window; always end with bsk session stop to clean up.
- Observation Priority: Default to bsk snapshot to capture page structure and @eN element refs; only escalate to bsk get-html or bsk screenshot when snapshots are insufficient.
- Interaction Loop: Operate elements in the Agent Window via commands like bsk click or bsk fill, using refs from the latest snapshot. For user tabs, first borrow them into the Agent Window with bsk tab borrow before writing.
- Security Boundaries: All writes are confined to the Agent Window or borrowed tabs; script execution (bsk evaluate) is only allowed in Agent tabs to mitigate cross-origin risks.

Key steps involve defining a success goal first, then following a tight path of snapshot→action→check to avoid redundant exploration. When encountering captchas or logins, use bsk request-help to pause and request user intervention.

Usage Considerations

• Applicable Scenarios: Tasks requiring real browser context, such as automating multi-step forms, UI smoke tests, or replaying user operation traces (trace.json).
• Prohibited Actions: Never run bsk evaluate on banking, SSO, or password manager pages to extract credentials; do not long-borrow user tabs; stop the session immediately after goal achievement without additional verification.
• Error Handling: Use diagnostics like bsk doctor based on exit codes (e.g., 1 for argument errors, 4 for timeouts); re-snapshot after navigation to refresh element refs.
• Recording Feature: Capture user actions in the Agent Window via bsk record into a trace.json for later automation, but avoid recording on sensitive pages.