dsh-llm-verifier

In DeepSeek Harness, the main Agent focuses on executing specific tasks like writing code or running commands, but often lacks an independent verification mechanism to ensure quality and reliability. dsh-llm-verifier fills this gap by introducing a 'player + referee' model: the main Agent acts as the player performing work, while the Verifier serves as the referee, collecting task goals, candidate solutions, and actual execution results, then passing them to a separate model you specify for arbitration. This helps assess which solution is more reliable, the actual progress of tasks, and any hidden errors, thereby enhancing the rigor of automated workflows.

The plugin's core capabilities include four explicit tools for pairwise comparison, tournament selection, progress tracking, and session verification, covering the entire process from candidate screening to task acceptance. It supports smart or strict auto-routing strategies that automatically schedule these tools at lifecycle boundaries of Agent sessions, combined with an A–T fine-grained scale and probability scoring mechanisms for precise evaluation. Additionally, through the Probabilistic Pivot Tournament algorithm, it efficiently handles multiple candidate solutions, significantly reducing API call costs. All configurations can be visually adjusted in DSH settings, including model selection, reasoning effort, and caching policies, while ensuring session data is anonymized for privacy.

dsh-llm-verifier is particularly suitable for teams and individuals using DeepSeek Harness for complex task development, such as code generation, multi-step workflows, or long-term project monitoring. Whether you need automated quality validation, comparison of different solutions, or real-time task progress tracking, it provides reliable referee support, helping to reduce manual review burdens and improve development efficiency and delivery quality.