Preface

DeepSeek Harness (DSH) uses a plugin-based architecture, allowing users to extend functionality. When debugging or validating model inference, developers sometimes need to verify that the requested model matches the model declared in the upstream response. DSH Model Audit is a standalone local plugin used to compare the requested model with the upstream response model.

What Is This

This is an unofficial local plugin maintained by jghwwnq under the MIT license. It runs in the DSH host environment and can read and compare the outgoing request model with the upstream response model.

Core Features

The plugin mainly provides the following capabilities:

  1. Model comparison: Compares the outgoing request model with the upstream response model.
  2. Protocol support: Supports DeepSeek Messages, GPT Responses, and Chat Completions.
  3. Platform compatibility: Works in both the Web and Desktop versions.
  4. Call monitoring: Monitors llm/stream and global fetch calls.
  5. Metadata retention: Preserves metadata such as model identifiers, response identifiers, usage, timing, and call purpose.

Installation and Activation

The plugin is not published to npm and is used only as a local package.

Web Version Installation

In the Web version, add the plugin directory via the CLI:

dsh plugin --profile web add /absolute/path/to/dsh-model-audit --offline --ignore-scripts

Desktop Version Installation

The desktop application exclusively owns its profile, and the CLI rejects --profile desktop. Add the absolute directory path on the desktop application’s plugins page (Installed plugins).

Activation Steps

  1. After installation, restart DSH once to register dependency resolution.
  2. Enable Model monitor in the DSH interface.
  3. Open its details panel and select Open monitor.
  4. Subsequent toggle changes are applied in real time.

Typical Usage

After installation and activation, the plugin records call data in memory.

  • Web version: Open monitor opens the dashboard in a new browser tab.
  • Desktop version: The dashboard is rendered inside the application window (because popups are disabled in the desktop shell); close it using the header button or the Escape key.
  • Data retention: Records are kept in memory only and are automatically cleared when the plugin is disabled or DSH is restarted.

Use Cases and Cautions

This plugin is suitable for developers who need to validate model mapping.

Notes:
* The plugin version 0.1.7-rc.2 targets DSH 0.1.7-rc.2; version mismatches may cause loading failures.
* Monitors specific calls only: The plugin only monitors llm/stream and global fetch. WebSocket, private transports, and replay mode are not observed.
* Data scope: The dashboard shows only model/response identifiers, usage, timing, and call purpose metadata; it does not include prompts, completions, or credentials.
* Host-side changes: After installing the plugin in the desktop version, modifying host plugin configuration still requires restarting DSH.

Conclusion

DSH Model Audit provides a lightweight local mechanism to verify model declarations. It is suitable for local debugging and validation of model inference pipelines. For more details and source code, see the GitHub repository.