Introduction

DeepSeek Harness (DSH) adopts a plugin-based architecture. Agent skills, tool wrappers, prompts, and policies usually require manual maintenance. dsh-auto-evolve is a self-evolving plugin. It observes runtime signals, uses an LLM to propose improvements, validates them in a sandbox, and finally applies only validated, versioned changes. It also supports automatic rollback.

Core Functionality

The plugin runs within the DSH process context and mainly includes the following stages:

  1. Observation: Listens for tools/result and agent/request-error signals, records data such as tool failures, repeated calls without progress, and request errors, and starts the evolution process when thresholds are triggered.
  2. Proposal: Based on the genome (skills, post-processors, prompt snippets, and policies) and observation data, it uses an LLM to generate change candidates. The change content is strictly validated with Zod schemas, and syntactically invalid generations are rejected.
  3. Validation: Reproduces failure scenarios in a sandbox and compares metrics between the baseline and the state after applying the change (completion rate, tool-call cost, etc.). It passes only when improvement is demonstrated.
  4. Application and Versioning: Validated changes are registered as skills or policies. The plugin maintains a versioned ledger that records the before and after state of each change.
  5. Rollback: If a regression occurs after applying a change, immediate rollback to the previous version is supported. Rollback is a first-class feature.
  6. Cost Control: Limits per-cycle and daily budget caps to prevent excessive Token consumption during validation.

Installation and Enablement

The plugin is provided as a DSH Bundle and does not require manual editing of cordis.yml. The installation commands are as follows:

# 安装到 web 配置文件（默认 UI 配置）
dsh plugin --profile web add dsh-auto-evolve

# 安装到 TUI 配置文件
dsh plugin --profile tui add dsh-auto-evolve

For local development or source builds:

git clone https://github.com/lispking/dsh-auto-evolve.git
cd dsh-auto-evolve
pnpm install
pnpm build
# 安装本地构建产物
dsh plugin --profile web add /absolute/path/to/dsh-auto-evolve

Configuration and Modes

After installation, the plugin automatically loads the default configuration. If customization is required, the relevant lines in the configuration file must be overridden. Note: Patch mode replaces an entire configuration line, so all fields that should be retained must be specified.

Configuration file path example: ~/.dsh/profiles/web/cordis.patch.yml

- id: self-evolve
  config:
    mode: auto-apply        # observe | propose | auto-apply
    observation:
      toolFailureThreshold: 3
      repeatThreshold: 3
      requestErrorThreshold: 3
      windowMs: 300000
    proposal:
      maxProposalsPerTrigger: 1
      maxEpisodesPerProposal: 5
      maxPromptChars: 24000
      maxTokens: 2000
    budget:
      maxCostPerCycle: 0       # 0 表示禁用单循环上限
      dailyBudget: 0           # 0 表示禁用每日上限
    validation:
      maxTrialMs: 30000
      maxToolCalls: 20
      maxTrialSteps: 12
      maxTrialTokens: 8000

Evolution Modes

Mode Behavior
observe Collects signals and triggers thresholds only; no proposals are made. Default mode.
propose When thresholds are reached, generates candidate changes and persists them, waiting for manual approval.
auto-apply Runs the full closed loop: observe -> propose -> validate -> apply. Automatic rollback occurs if a regression is detected.

Operational Tools

The plugin registers the following tools for the agent. They can be used to check status or intervene manually:

  • evolve_status: View a health summary, including mode, generation, asset state, ledger/observation totals, convergence pause status, and more.
  • evolve_candidates: List candidate changes waiting for validation or manual application.
  • evolve_apply: Manually approve and apply a candidate change.
  • evolve_rollback: Roll back an applied asset and restore it to candidate state.
  • evolve_cycle: Manually trigger one evolution cycle.

Security and Notes

  • Execution-based Validation: Whether a change takes effect depends on execution results in the sandbox, not on the LLM’s self-report.
  • Vocabulary Constraint: The LLM generates content only within a predefined closed vocabulary (such as add / patch / retire) and does not create new asset types.
  • Runtime Permissions: The plugin runs with the permissions of the current DSH process. Before installing, it is recommended to review the source code to confirm that its behavior matches expectations.
  • License: MIT.

Summary

dsh-auto-evolve addresses the repeated debugging problem in agent asset maintenance and reduces maintenance costs through an automated closed loop. It provides full-chain capability from observation to application and ensures stability through sandboxing and cost control.