Introduction¶

When connecting a browser to an agent, two common approaches each have their trade-offs: letting the model run a headless browser, where the user cannot see what it clicked; or having the user browse themselves and copy the content to the model. dsh-playwright takes a third path—opening a Browser Use panel in DeepSeek Harness (DSH), where you and the model see and operate on the same Playwright page. Below introduces the plugin’s positioning, features, installation, and usage.

What is it¶

dsh-playwright is a DSH plugin maintained by JeremyGuo and licensed under MIT. One-sentence positioning: using a shared Playwright browser in conjunction with your DeepSeek Harness AI. DSH’s philosophy is that everything is a plugin, and browser capabilities are integrated via plugins: each DSH session has its own independent BrowserContext and Page. The model navigates, reads, clicks, types, and takes screenshots via a set of browser_* tools, and what you see in the panel is the same real-time view.

Core Features¶

Browser Use Panel¶

Chrome CDP pushes repaint frames via a loopback-only binary WebSocket to a stable canvas. The panel shares width with the chat, and the left divider is draggable with a range of 10%–80%. When exceeding 55%, the plugin collapses the DSH left Workspace/session selector; when below 48%, it only restores the selector it collapsed itself.

Navigation capabilities cover opening HTTP(S) URLs and refreshing the current page. browser_navigate opens the URL and returns a semantic snapshot; browser_snapshot returns the bounded page text, along with temporary IDs of interactive elements such as visible links, buttons, and input fields (e.g., e1). Note that these IDs are only valid for the current snapshot; they need to be re-acquired after navigation or dynamic UI changes.

Pointer and Keyboard¶

browser_click defaults to left-click and can target by element ID or screenshot coordinates. It supports left, right, and middle mouse buttons with actions like click, down, move, and up. Dragging consists of down → one or multiple moves → up, using the same mouse button. browser_type replaces the value of visible, editable elements and optionally submits with Enter; browser_keyboard supports type, press, down, and up, including modifier combinations like Control+A and Meta+K, and long-pressing game keys with down/up.

Screenshot and Click Marking¶

browser_screenshot returns PNG via DSH’s native attachment and image block pipelines, limited to model routes that declare image input. It can mark the location of the last successful click on the screenshot: the marking is a post-processed RGB square, does not modify the page DOM, and has an independent “Show Last Click” toggle in the panel.

Session Isolation and Recovery¶

Chrome launches lazily; during teardown, queued operations complete before closing. If Chrome exits unexpectedly, the next operation will launch a replacement browser and restore the affected session to the last top-level URL.

Installation and Enablement¶

First, confirm your environment meets the requirements:

DeepSeek Harness 0.1.0-rc.6
Node.js ^22.19.0 or >=24.0.0
Host installed with Google Chrome or Chromium
To use browser_screenshot, the model route must support image input

The installation commands provided in the README are as follows (labeled “After publication”, meaning the installation method after official release):

dsh plugin --profile web add dsh-playwright
dsh web --port 3081

For local development, you can mount a local directory using the link method. First, pull dependencies and run type checking, tests, and build:

pnpm install
pnpm run verify
dsh plugin --profile web add link:/absolute/path/to/dsh-playwright
dsh web --port 3081

After enabling, use the Browser Use button at the right edge of the interface to toggle the panel, and drag the left divider to adjust the ratio.

Typical Usage¶

Open a page and get a semantic snapshot:

{"url":"https://example.com"}

Clicking can use the element ID from the snapshot or coordinates:

{"element_id":"e1"}

{"action":"click","button":"right","x":640,"y":400}

Move the pointer without clicking, steps controls the intermediate steps:

{"action":"move","x":640,"y":400,"steps":20}

Dragging is sent in three steps, up and down use the same mouse button:

{"action":"down","button":"left","x":200,"y":200}
{"action":"move","x":600,"y":400,"steps":30}
{"action":"up","button":"left","x":600,"y":400}

Input text and submit with Enter:

{"element_id":"e3","text":"DeepSeek Harness","submit":true}

Keyboard operations support typing, combinations, and long presses:

{"action":"type","value":"hello","element_id":"e3","delay_ms":50}

{"action":"press","value":"Control+A"}

{"action":"down","value":"ArrowRight"}
{"action":"up","value":"ArrowRight"}

When taking a screenshot, you can optionally mark the last successful click to easily verify if the click landed in the expected position:

{
  "show_last_click": true,
  "marker_r": 0,
  "marker_g": 255,
  "marker_b": 0,
  "marker_size": 60
}

The three RGB channels are integers from 0–255, and show_last_click defaults to false. The README mentions that the plugin will guide the model to proactively refresh stale element IDs at the prompt level, switch to screenshot coordinates on canvas-like pages, release mouse and keyboard inputs during long presses, and request marked screenshots when verifying clicks.

To use the screenshot feature, you must declare image input for the model route. Taking the llm-pi-ai route as an example:

llm-pi-ai:
  providers:
    my-provider:
      models:
        - id: my-vision-model
          name: My Vision Model
          input:
            - text
            - image

Declaring image input is a declaration of endpoint capabilities; if the endpoint does not actually accept images, the provider will reject the request.

Configuration Options¶

browserExecutablePath: Explicitly specify the Chrome/Chromium executable; omitted to use common macOS/Linux paths
viewportWidth / viewportHeight: Shared page viewport
navigationTimeoutMs: Playwright operation timeout
actionSettleMs: Delay before returning the status after an operation
screencastQuality: CDP screenshot JPEG quality, ranging from 30–90, default 60
maxSnapshotChars / maxSnapshotElements: Volume limit for semantic snapshots
allowPrivateHosts: Set to true to allow localhost and private network targets, default false

Security Boundaries¶

Several design points worth understanding first:

RPC and screencast endpoints only accept loopback same-origin clients.
By default, HTTP(S) requests to localhost, literal private addresses, and hostnames resolving to private addresses are intercepted, including sub-resources. allowPrivateHosts is only enabled for trusted tasks.
Private address interception is network protection, not a complete sandbox against untrusted websites.
Browser sessions do not reuse user Chrome profiles, cookies, or login states, and popups and downloads are not exposed externally.

Suitable Scenarios and Notes¶

Suitable occasions: Tasks where the model needs to operate on real web pages in DSH—seeing, clicking, and inputting—while the user wants to oversee the entire process and take over at any time, such as debugging an agent’s web interaction flow.

Notes before use:

The plugin runs with the permissions of the current dsh process; check the source code and license before installing (this project is MIT).
Element IDs in snapshots are only valid for the current snapshot; the model needs to re-acquire them after navigation or dynamic page changes.
browser_screenshot is only available when the model route declares image input.
As mentioned earlier, allowPrivateHosts should only be enabled for trusted tasks.

Conclusion¶

dsh-playwright puts “what the user sees” and “what the model can operate on” into the same page, saving the cost of bouncing back and forth between headless browsers and manual copy-pasting. If you are using DSH for agent development, it is worth a try.

GitHub: https://github.com/JeremyGuo/dsh-playwright
Community Directory: https://www.skillhub.cn/plugins/JeremyGuo/dsh-playwright (Independent community site, no official affiliation with DeepSeek / Matrix Factor)

dsh-playwright: Share a browser page with the model in DeepSeek Harness