In DeepSeek Harness (DSH), relying only on a prompt instruction such as “do not guess” can neither be enforced nor audited. dsh-evidence-gate structures conclusion discipline by enforcing evidence hierarchy and cross-checking, preventing agents from drawing conclusions from unverified guesses and ensuring that every conclusion is auditable.

Plugin Information

  • Name: aaronandwork/dsh-evidence-gate
  • Maintainer: AaronandWork
  • License: MIT
  • Core Value: Make agents in DeepSeek Harness (DSH) stop drawing conclusions from unverified guesses — and make every conclusion auditable.

Core Features

The plugin provides a structured conclusion verification mechanism:

  1. 3-tier evidence hierarchy policy: Establishes an evidence hierarchy policy and manages conclusions by category.
  2. Model tool verdict: Registers the evidence_gate tool to evaluate statements made by agents (VERIFIED/REFERENCED/UNVERIFIED).
  3. Persistent session-level cross-checking: Persistently records tool activity within a session and performs cross-verification based on parameters and output anchors.
  4. Session-level switch and ledger: Provides a session-level switch and a persistent statistics ledger.
  5. UI interaction: Provides a session-level UI toggle (shield chip).
  6. False-positive mitigation ladder: Reduces false positives through multiple decision layers.
  7. Output-aware cross-checking: Cross-checking matches not only command strings but also keywords, paths, and URLs in tool outputs.
  8. Silent startup: No output by default when mounted; information is shown only in debug mode.
  9. Honest-claim protection: When verification is not possible, allows pass-through for references to paths that actually exist (with a weak annotation) to prevent false positives.

Installation and Activation

The plugin is published as a DSH Bundle and does not require manual configuration editing.

pnpm add github:AaronandWork/dsh-evidence-gate

After installation, it must be enabled in the configuration:

"bundles": [ "@aaronandwork/dsh-evidence-gate" ]

Notes:
* The pnpm add command rewrites package.json; manually check the bundles configuration.
* Ensure that the files whitelist in package.json includes cordis.patch.yml.
* After installation, restart the deployment; it loads automatically after the browser is refreshed.

Typical Usage

The plugin is controlled through command-line instructions and UI interactions.

  1. Command-line control:

    • /evidence-gate on: Enables the evidence gate for the current session.
    • /evidence-gate off: Disables the evidence gate for the current session.
    • /evidence-gate status: Shows the current session status, ledger, verification hit count, and index size.
    • /evidence-gate report: Generates a complete session report, including verdict groups, basis statistics, and blocked/converted/accepted data.

  2. UI interaction:
    In the editor interface, use the shield chip icon to toggle the switch; the switch applies only to the current session.

Working Mechanism

  • Policy injection: Injects policy text through systemPrompt.section (order 550).
  • Evidence hierarchy: Defines hierarchy levels such as tested/measured/read_directly/prior_verified (direct evidence) and documented/community_practice (referenced sources).
  • Persistent storage: The ledger, switch state, and evidence index are stored in DSH_HOME/evidence-gate/store.json and preserved across restarts.
  • Cross-checking: Listens to the tools/result stream and records tool parameters and output anchors. Before stating a key conclusion, the agent must call the evidence_gate tool.

False-positive mitigation logic (ladder):
1. Hit in the current session
2. Hit on recorded output keywords
3. Hit in another session (sub-agent)
4. Existence fallback (the referenced path actually exists and passes with a weak annotation)
5. Block

Applicable Scenarios and Notes

  • Applicable scenarios: Scenarios that require strict auditing of agent conclusion sources, preventing hallucinations, and building high-confidence agents.
  • Permission note: The plugin runs with DSH process privileges; review the source code and license before installation.
  • Debug note: Set the environment variable EVIDENCE_GATE_DEBUG=1 to view startup logs and loading information.