Introduction

When developing the DeepSeek Harness agent, enabling web operation capabilities usually requires calling cumbersome browser automation interfaces or asking users to manually operate multiple windows, which is inefficient. The dsh-inapp-browser plugin provides a built-in browser panel, allowing the AI and the user to operate web pages within the same context. It supports the complete workflow from opening URLs and interacting with elements to taking screenshots and reading page content.

What Is This

dsh-inapp-browser is a client plugin maintained by developer alllllinnrann. It uses the Chrome DevTools Protocol (CDP) to render a standalone browser instance inside Better Sidebar. By default, the browser starts on the Bing search page, and user login profiles are saved per chat session, preventing cross-session interference.

Core Features

The plugin provides a rich set of tools that covers the main scenarios for web browsing and interaction.

Browser Control

  • Tab Management: Supports multi-tab operations. Use iab_tabs to view the current tab list, iab_select to switch tabs, and iab_close_tab to close a tab (at least one tab must be retained).
  • Navigation and Refreshing: Supports opening URLs or search terms with iab_open (supports Bing, Baidu, and Google engines), going back with iab_back, going forward with iab_forward, and reloading the page with iab_reload.
  • Dialog Handling: Provides the iab_dialog interface to confirm or cancel JavaScript web page dialogs, such as alert, confirm, and prompt.

Page Interaction

  • Clicking and Typing: Supports mouse clicks with iab_click (left, right, and middle buttons can be specified), typing Chinese text into the focused control with iab_type, and drag operations with iab_drag.
  • Scrolling and Key Input: Supports page scrolling with iab_scroll (direction and position can be specified) and simulated keyboard key presses with iab_key (supports combinations, such as Select All with Ctrl+A).
  • Waiting Mechanism: Provides the iab_wait interface to wait for page loading before performing the next operation, with a range of 0–10000 milliseconds.

Information Reading

  • Page Snapshot: Calling iab_snapshot retrieves the current page URL, title, text content, control list (including position, labels, and whether the control is within the viewport), and viewport size.
  • High-Quality Screenshot: Calling iab_screenshot retrieves an image attachment of the current page and viewport information. Note that screenshots require a host and model that support image input.

Installation and Dependencies

Before using this plugin, ensure the environment meets the following version requirements:

  • DeepSeek Harness: 0.1.5-rc.2
  • Node.js: 22.19 or later
  • Better Sidebar: 0.19.1 through 0.19.x

Typical Usage

The following are typical tool call examples provided by the plugin. The tools automatically use the current chat session and do not require a chat ID to be passed in.

Open a URL and Search

{
  "name": "iab_open",
  "arguments": {
    "url": "https://www.bing.com",
    "engine": "bing",
    "newTab": false
  }
}

Type Text

{
  "name": "iab_type",
  "arguments": {
    "text": "Hello World"
  }
}

Click Coordinates

{
  "name": "iab_click",
  "arguments": {
    "x": 100,
    "y": 200,
    "button": "left"
  }
}

Screenshot

{
  "name": "iab_screenshot"
}

Configuration and Notes

Environment Variables

You can customize the browser path and data directory by setting environment variables:

  • DSH_IAB_BROWSER_PATH: Specifies the browser executable path. By default, it automatically searches for Edge, Chrome, or Chromium.
  • DSH_IAB_DATA_DIR: The browser profile root directory. The default is ${DSH_HOME}/storages/dsh-inapp-browser/.
  • DSH_HOME: The Harness data root directory. The default is .dsh under the user directory.
  • DSH_IAB_HOME_URL: The home page for new sessions. The default is https://www.bing.com/.

Coordinate and Key Rules

  • Coordinate System: The x and y parameters in iab_click and iab_drag are CSS pixels in the web page viewport, not coordinates scaled in the sidebar image. If the model side scales the screenshot, the values must be converted back to the original viewport coordinates.
  • Key Combinations: The modifiers field in iab_key uses bitmask combinations: Alt=1, Ctrl=2, Meta=4, Shift=8. For example, Select All is represented as {"key": "a", "modifiers": 2}.

Other Limitations

  • Screenshot functionality depends on a model and host attachment service that support image input.
  • The plugin does not take over the browser already open for the user’s daily use. It runs in a separate process and is used only for the current Harness chat session.

Summary

dsh-inapp-browser provides native browser control capabilities for DeepSeek Harness through the CDP protocol. It solves the context fragmentation problem when AI operates web pages. By sharing login state and providing a visible panel, it significantly lowers the barrier to web automation development. For more details, see the GitHub repository.