dsh-plugin-judge

In the DeepSeek Harness plugin ecosystem, constraint-type plugins carry a hidden cost that is easy to overlook: they make the model obey rules faithfully, yet simultaneously squeeze the context budget away from the task reasoning that actually matters, capping a capable model from below. The situation worsens with every model upgrade—rules written as patches for an older model flip into ceilings, and a once-helpful plugin quietly becomes harmful. dsh-plugin-judge addresses exactly this gap by treating a plugin's value as a binary relation between the plugin and the current model, rather than a fixed property of the plugin itself.

It provides a closed loop across three workflow stages. Before installation, the /plugin-audit command, the judge_plugin tool, or the settings-page input fetches a target plugin's source, runs a static scan plus an LLM judgment, and returns a "worth installing?" verdict. After installation, it enumerates every bundled plugin in the active profile, classifies each as capability, constraint, cosmetic, or hybrid, computes a 0-100 constraint-risk score, and renders a full report in the Plugin Judge panel. When the default model changes, it detects the switch, flags plugins whose earlier verdict depended on the old model, and surfaces a floating reminder prompting a re-audit.

The judgment engine works in two layers. The first is a deterministic rule heuristic that scans plugin source for signals such as tools.register, systemPrompt.section, complete: true wholesale replacement, and strong-directive density, then weights them into a constraint-risk score and assigns a category. The second layer, invoked on demand, hands the scan results, an injected-content summary, and the current model's identity to an LLM and asks a pointed question: do these rules help or suppress this specific model?—returning a verdict with reasons. Together the two layers are fast, explainable, and model-aware.

This plugin suits Harness administrators managing a growing number of plugins, especially those with a significant share of constraint-type entries or who track frequent model upgrades. It does not replace a security audit nor block any installation; it simply asks one extra question at three critical moments—before you install, after you install, and whenever the model under you changes—is this plugin still worth keeping for the you of today?