Introduction

DeepSeek Harness (DSH) adopts a plugin-based architecture, aiming to extend system capabilities through plugins. In agent development, enabling models with operating-system-level control (such as operating browsers or clicking desktop applications) is a common requirement. The dsh-computer-use plugin provides a unified interface, solving how models interact with external environments.

What Is This

dsh-computer-use is a DeepSeek Harness plugin maintained by xiaoheizi1212. It provides a set of “general-purpose” computer operation capabilities, not tied to a specific underlying model, and aims to offer DSH a model-agnostic browser and desktop operation interface.

The plugin uses the ctx.computerUse capability interface to encapsulate underlying browser drivers, native assistants, and perception logic, making them available for invocation by upper-layer models.

Core Features

The plugin provides the following core components:

  • Unified Interface: Provides the ctx.computerUse seam, including start, listTargets, observe, act, and stop methods.
  • Model Tools: Exposes 6 tool functions to the model:
    • computer_observe: Observe the current state.
    • computer_act: Execute an action.
    • computer_stop: Stop operations.
    • computer_take_over / computer_resume: Take over or resume a session.
    • computer_perceive: Perceive the environment (take a screenshot and analyze it).
  • Multi-Backend Support: Provides three Providers:
    • fake: Used for deterministic testing.
    • playwright: Cross-platform isolated Chromium browser.
    • windows: Windows native assistant (Windows only).
  • Perception Layer:
    • Accessibility Mode: Does not generate images; analyzes only through the accessibility tree.
    • Analyze Mode: Sends a screenshot to a third-party vision model for structured analysis.
  • Safety Mechanism: Includes risk classification, fail-closed confirmation mechanism, domain allowlists, and full session log replay for computer/*.
  • Cookie Bridge: Supports importing logged-in Chrome sessions.

Dependencies and Configuration

The plugin depends on several base plugins in the DSH ecosystem when running. In the configuration file, all options are located under dsh-computer-use/plugin.

Required Plugin Dependencies

The plugin requires the following DSH plugins to work:
* @deepseek-ai/dsh-llm-pi-ai: Handles third-party vision routing.
* @deepseek-ai/dsh-llm: Processes image content blocks.
* @deepseek-ai/dsh-attachment: Persists screenshots.
* @deepseek-ai/dsh-credentials: Parses vision API keys.
* @deepseek-ai/dsh-user-approval: Handles confirmation for high-risk operations.
* @deepseek-ai/dsh-tools: Tool registration.
* @deepseek-ai/dsh-llm-deepseek: Used only as a text planning model, and cannot be used as a vision model.

Configuration Example

Configure the plugin in cordis.patch.yml:

- id: computer-use
  name: dsh-computer-use/plugin
  config:
    # 后端选择：fake, playwright, windows
    provider: playwright
    # 视觉模型配置（需在 llm-pi-ai 路由中配置）
    visionProvider: xiaomi
    visionModel: mimo-v2.5
    # 是否启用主开关
    enabled: true
    # 是否注册模型工具
    tools: true

You can also modify configurations such as enabled and provider in real time through the Settings -> computer-use menu in the DeepSeek Harness GUI.

Playwright Browser Session Configuration

When using the playwright provider, you can configure the following parameters:

- id: computer-use
  config:
    provider: playwright
    browserHeadless: true
    browserWindowState: normal
    # 使用持久化而非隔离上下文
    reuseBrowserProfile: true
    browserUserDataDir: "C:/Users/you/.dsh/browser-profiles/main"
    browserProfileName: "Default"
    # 导入 Cookies（需要配合 Cookie Bridge）
    importCookies: true
    cookiesFile: "C:/path/to/cookies.json"

Windows Native Assistant Configuration

When using the windows provider:

- id: computer-use
  config:
    provider: windows
    windowsWindowState: normal

Typical Usage: Cookie Bridge

By default, the browser is isolated and has no Cookies. For scenarios that require a logged-in state, the Chrome Cookie Bridge is recommended.

Enable developer mode in the Chrome browser and load the unpacked extension (see the GitHub dsh-cookie-bridge repository for reference).

2. Start the Receiver Script

Run the receiver script in the plugin directory to receive Cookies read from Chrome:

pnpm exec tsx scripts/import-cookies-server.ts

3. Export Cookies

Click the extension icon or export by specifying domains through the script:

# 导出指定域名
pnpm exec tsx scripts/request-cookies.ts all x.com,xiaohongshu.com

Save the exported JSON file as cookies.json, then point the plugin configuration to it:

- id: computer-use
  config:
    importCookies: true
    cookiesFile: "C:/path/to/cookies.json"

Notes

  1. Platform Limitation: The Windows native helper only supports the Windows platform. macOS is not currently developed and cannot be used.
  2. Vision Model Selection: @deepseek-ai/dsh-llm-deepseek only supports text and cannot be used as the vision model for computer_perceive.
  3. Chrome Version: Chrome 127+ introduced App-Bound Encryption. Cookies must be imported using the Cookie Bridge; otherwise, a logged-in state cannot be obtained.
  4. Configuration File Isolation: Reusing the default Chrome User Data directory is not supported, and directly copying the default configuration profile is also not supported (App-Bound Encryption prevents this). Only the dedicated configuration profile managed by the plugin can be used.
  5. Security Risk: Importing Cookies, passwords, or browsing history weakens isolation. Import only data required by the task, and ensure that the allowedDomains allowlist is correctly configured.