Preface¶
When using a single agent for review, a common problem is that preparation materials and scoring are handled within the same context, making scores easily swayed by previous discussions. Even with multiple sessions, roles are not isolated, and there are no forced checkpoints to guarantee that every step is actually completed. For developers building agent applications, ensuring that a “review” process is reproducible and traceable requires a fixed process combined with role isolation.
review-workflow turns this into a DSH (DeepSeek Harness) plugin, aligning with the DSH philosophy that “everything is a plugin.” Below is an introduction to its workflow, installation method, and the configuration required before integration.
What is this¶
review-workflow is a workflow-style DSH plugin maintained by LeslieWylie, version 0.1.0 in package.json, with an MIT license. In one sentence, it defines: N judges conducting blind reviews in isolated subagents, a Chair responsible for resolving disagreements, and an independent Critic auditing the entire process.
According to the README, it is generalized from the internal collaboration review process, having removed all internal project references. It is suitable for scenarios requiring structured reviews, such as project reviews, paper peer review, code review, and design review.
Core Process: 6 Steps + 6 Checkpoints¶
The entire process is divided into 6 steps, each with a checkpoint; progression to the next step is not allowed without passing the current one:
- Material Integrity Check
- Prepare Bundle (Review Material Package)
- N Judges Parallel Scoring
- Aggregate Disagreements
- Critic Review
- Final Decision
Several aspects of these steps are worth special mention:
- Blind Review with Judge Isolation: Each judge runs in an independent subagent and is invisible to others, preventing bias from influencing the judges.
- Two-Stage Disagreement Handling: During the aggregation phase, anchoring consistency checks are performed. First, anchoring matching is done, followed by \(\Delta\)-level resolution, which is then resolved by the Chair.
- Independent Critic Review: The Critic also runs in an independent subagent and independent session to review process compliance and challenge scores with insufficient justification.
- Incremental Re-review: Supports re-review scenarios where only dimensions related to modified items are re-evaluated, avoiding a full re-run.
- Auditability: All intermediate artifacts are saved and traceable.
Installation and Activation¶
Installation command:
dsh plugin --profile web add "github:LeslieWylie/review-workflow"
After restarting, load the skill in any session:
load review-workflow
Runtime environment requirements: Node >= 18 (as specified in the engines field of package.json).
Configuration Preparation Before Integration¶
Note that this plugin only defines the process skeleton and does not come with built-in review standards. Before integration, project-level configuration needs to be prepared and placed in the project’s own skill or configuration file:
| Configuration Item | Description | README Example |
|---|---|---|
| rubric | Review dimensions and scoring criteria | 5-dimension 5-point scale, each dimension has anchoring descriptions |
| bundle template | Review material package template | bundle.md |
| panelist prompt | Judge prompt template | panelist-prompt.md |
| aggregation template | Aggregation report template | aggregation.md |
| critic template | Review report template | critic-report.md |
| checklist | Material integrity checklist | proposal-checklist.md |
| panelist count | Number of judges | 2 |
| scoring thresholds | Pass/Fail thresholds per dimension | ≥3/5 |
These configurations directly determine the quality of the review. It is worth spending time detailing them during integration, especially the anchoring descriptions and scoring thresholds in the rubric.
Design Principles¶
The README lists four design principles:
- Scenario-Agnostic: It only defines the review process skeleton; specific review dimensions, scoring criteria, and templates are provided by the project.
- Role Isolation: Judges run independently, and the Critic uses an independent session to prevent bias.
- Checkpoint Enforcement: Every step has a checkpoint; progression is blocked if a step is not passed.
- Auditability: All intermediate artifacts are saved and traceable.
Applicable Scenarios and Precautions¶
Suitable scenarios are reviews requiring multiple perspectives and conclusions that can withstand scrutiny, such as code review, paper peer review, design review, and project review. If you are simply letting a model provide opinions, this process might feel overkill; its value lies in hardening the easily deviating steps—such as judge isolation, disagreement resolution, and process compliance—into checkpointed steps.
Before installation, note three points:
- The plugin runs with the permissions of the current
dshprocess; you should check the source code and license before installing. The source code for this plugin is publicly available on GitHub under the MIT license. - It does not come with review standards; rubrics, templates, and checklists must all be provided by the project. Leave preparation time before integration.
- Confirm that the runtime environment meets Node >= 18.
Conclusion¶
review-workflow solves a specific problem: enabling reviews involving agents to have isolated roles, forced checkpoints, and traceable intermediate artifacts, while leaving “what to review” and “how to review” to be defined by the project. If your DSH project requires structured reviews, you can first compare its 6-step process and configuration checklist to see if it matches your scenario.
- GitHub Repository: https://github.com/LeslieWylie/review-workflow
- Community Plugin Directory: https://www.skillhub.cn/plugins/LeslieWylie/review-workflow (Independent community site, with no official affiliation to DeepSeek or Magic Form)