Introduction

In DSH’s plugin development process, a common requirement when comparing coding results from multiple models is to run each model’s modifications in isolation, verify them with a unified command, review the diffs one by one, and explicitly select which results to keep. dsh-agent-arena addresses this workflow by providing battles between 2–4 configured models, deterministic verification, scoring, and reporting, with the final result explicitly applied by the user.

What is it

dsh-agent-arena is an MIT-licensed DSH plugin maintained by LeemanCheung, positioned as “Isolated multi-model coding matches with deterministic verification, scoring, and reports”. It is suitable for DSH users who need to compare coding results from multiple provider/model routes and wish for the acceptance process to be independent of an LLM judge.

Core Features

Multi-model Isolated Matches

  • Supports comparing 2–4 configured models in isolated Git worktrees.
  • Each contestant starts via the public ctx.agents.create, SessionId, createUserMessage, and Agent.followup APIs.
  • Matches support Start, Cancel, diff review, and Apply winner, and do not rely on browser global objects.
  • The plugin reviews each diff and explicitly applies a winner.

Deterministic Verification

  • Git and validation argv are executed via ctx.subprocess.spawn without using a shell.
  • The validation field in Settings accepts conservative whitespace-separated argv, for example:
corepack pnpm test
git status --porcelain=v1
  • The following inputs are rejected: quotes, shell variables, pipes, redirections, command separators, backticks, newlines.
  • Scoring does not use an LLM judge: each validation has a positive weight, and the contestant score is the percentage of total positive validation weight for successful validation exits; ties use the contestant ID as a stable deterministic tie-breaker.

Diff Review and Application

  • A clean git status --porcelain=v1 is required before the match starts; worktrees are only created after this.
  • Review uses the binary-capable git diff HEAD, covering staged, unstaged, and intent-to-add files.
  • The winner worktree is preserved until the user explicitly applies it.
  • During Apply, a cleanliness check is run again to confirm HEAD still equals the recorded base revision, then the following is executed:
git apply --check
git apply --index --whitespace=error
  • The Arena will not automatically apply, commit, push, or rewrite history.

Persistence and Boundary Limits

  • Match reports are persisted via storageDomain.
  • The plugin polls the generated agentArena Remote namespace in Settings.
  • After the host restarts, interrupted nonterminal matches are marked as failed.
  • Cancellation depends on whether the selected provider follows the supplied abort signal.
  • Patches exceeding the 1 MB review/apply bound are rejected before mutation, rather than being truncated.

Installation and Enablement

Installation command:

dsh plugin --profile web add github:LeemanCheung/dsh-agent-arena

After installation, the existing DSH Web process needs to be restarted and its page refreshed.

The selected profile must provide:

  • agents
  • subprocess
  • storageDomain
  • Typert Remotes
  • Settings
  • At least two available provider/model routes

Typical Usage

  1. Configure validation commands. Parameters should not require shell quoting, for example:
corepack pnpm test
git status --porcelain=v1

  1. Before starting the match, confirm the repository state satisfies a clean git status --porcelain=v1.

  2. Run the plugin-supported Start, Cancel, diff review, and Apply winner workflow to view each contestant’s diff, score, and report.

  3. If local development of the plugin is needed, run in the repository root:

corepack pnpm typecheck
corepack pnpm test
corepack pnpm build
corepack pnpm pack:check

Applicable Scenarios and Notes

Suitable for developers who need to compare multiple model coding results in DSH and wish for the verification, scoring, and application process to be interpretable and reproducible.

Notes before use:

  • The plugin runs with the current dsh process permissions; the source code and MIT license should be checked before installation.
  • Validation argv uses a deliberately conservative tokenizer and cannot express parameters containing whitespace.
  • Apply requires a clean git status --porcelain=v1 and that HEAD equals the recorded base revision.
  • Cancellation effectiveness depends on whether the selected provider follows the supplied abort signal.
  • Patches exceeding the 1 MB review/apply bound are rejected, not truncated.
  • The Arena will not automatically apply, commit, push, or rewrite history.

Conclusion

dsh-agent-arena’s purpose is to integrate multi-model coding comparison into a workflow featuring isolated worktrees, deterministic validation, diff review, and explicit application. GitHub repository: https://github.com/LeemanCheung/dsh-agent-arena