In scenarios involving model inference or experimental comparisons, determining whether the difference between two conversion rates is statistically significant often requires writing or incorporating dedicated statistical scripts. DeepSeek Harness (DSH) emphasizes “everything is a plugin,” extending the toolchain through a plugin mechanism. The dsh-ctr-significance plugin addresses this computational need, enabling precise calculations locally without network dependencies.

Plugin Positioning

This is a DSH plugin focused on model inference performance evaluation, maintained by developer uckkk. It provides conversion-rate significance calculation capabilities, allowing users to quickly obtain detailed statistical results locally to validate performance differences in experiments or model tuning.

Core Capabilities

  1. Precise per-parameter calculation: Calculates precisely by parameter and returns detailed results.
  2. No network dependency: Pure Node implementation that can run without internet access.

Installation

Install the plugin using DSH’s package manager:

dsh plugin add github:uckkk/dsh-ctr-significance

After installation, the tools registered by the plugin must be invoked in the current session.

Dependencies

The plugin depends on the following environment:

  • @deepseek-ai/cordis (^4.0.1)
  • @deepseek-ai/dsh-tools (>=0.1.0-rc.6)

Usage

Once installation is successful, directly invoke the tool registered by the plugin in the session (the tool name is ctr_significance).

Notes

  • Runtime permissions: The plugin runs with the permissions of the current DSH process.
  • Code review: It is recommended to review the source code and license (MIT) before installation.

The plugin simplifies the statistical validation process through local calculations, making it suitable for developers who frequently need to compare conversion rates. For more details, see GitHub or the community directory.