Introduction

In an intelligent agent development system like DeepSeek Harness (DSH) where “everything is a plugin”, after adjusting an Agent Profile, two questions usually need answering: has this change actually made things better, and can I roll back to a verified version if it breaks? dsh-self-evolution puts these two questions into the same process: it allows the Agent Profile to self-iterate on a frozen Benchmark, only accepting the candidate version if it is strictly better than the current version, otherwise rolling back precisely from a verified snapshot.

What is it

dsh-self-evolution is an MIT-licensed plugin maintained by Lhy723, targeting DeepSeek Harness / Cordis. It natively calls ctx.subagents.start(...) and ctx.tools.register(defineTool(...)), exposing itself as ctx.evolution as a Cordis Service, and registers the following model tools:

  • evolution_run
  • evolution_evaluate
  • evolution_status
  • evolution_rollback

Core Features

The following describes capabilities within their boundaries.

  • Iterates the Agent Profile on a frozen Benchmark.
  • Any improvement must be strictly better than the current version to be accepted; otherwise, it rolls back precisely from a verified snapshot; the acceptance rule does not use >=, ties are rolled back.
  • The host code executes deterministic strategies, including frozen Benchmark hashes, Case × Run matrices, strict acceptance thresholds, monotonically increasing version numbers, and snapshot verification.
  • Uses three isolated roles: Target, Evaluator, and Optimizer; the Optimizer uses { allow: [] } for full tool isolation.
  • Private rubric does not enter the Target / Optimizer context; public and private artifacts are stored in separate directories.
  • Change protection includes candidate file whitelists, path traversal and symlink protection, single Profile mutex locks, and detection of external modifications.
  • Each sub-agent has an independent DeepSeek Session ID; the Scoreboard and runtime event logs can be used for review.

Installation and Usage

Confirm prerequisites first:

  1. An initialized DeepSeek Harness profile already exists.
  2. The dsh CLI and pnpm are in your PATH.
  3. Node.js meets >=22.
  4. The Benchmark must declare frozen: true.
  5. In production, please place the Benchmark (especially the private rubric) outside the Target workspace; the default allowBenchmarkInsideWorkspace: false will reject Benchmarks inside the workspace.

To install in the target profile:

dsh plugin --profile web add dsh-self-evolution

After installation, view the current profile configuration:

dsh --profile web --dump-config

After completing the steps above, the plugin is registered as the ctx.evolution service and provides four evolution_* model tools during the session.

Typical Usage

The following performs an evaluation first, then runs an evolution round, followed by querying the status and rolling back.

Evaluate

Call evolution_evaluate:

evolution_evaluate(profile_dir, benchmark_dir, runs_per_case)

Here, benchmark_dir should point to a Benchmark directory that declares frozen: true.

Run Evolution

Call evolution_run:

evolution_run(profile_dir, benchmark_dir, rounds, runs_per_case)

The optional parameter target_score represents the target score:

evolution_run(profile_dir, benchmark_dir, rounds, runs_per_case, target_score)

Query and Rollback

Call evolution_status to query the current version, actual hash, drift, next version, Benchmark reference, and snapshot.

Call evolution_rollback to restore to a verified version snapshot.

Applicable Scenarios and Notes

Suitable for DSH users who maintain Agent Profiles and wish to verify modifications on a fixed Benchmark. Please note:

  • The plugin runs with the current dsh process permissions; check the source code and license before installing.
  • The Optimizer is restricted to { allow: [] } full tool isolation by default, and can only propose candidates via restricted file operations.
  • By default, the Optimizer is forbidden from modifying provider and model; set allowModelRouteMutation: true to enable this.
  • When installing from GitHub, pre-built dist is submitted directly; no prepare build step is needed, nor is allowBuilds configuration required.

Related Links

  • Plugin Directory Page: https://www.skillhub.cn/plugins/Lhy723/dsh-self-evolution
  • GitHub Source Code: https://github.com/Lhy723/dsh-self-evolution