quick-research-campaign: DeepSeek Harness Research Pipeline

Introduction¶

DeepSeek Harness (DSH) uses a plugin-based architecture, allowing community developers to extend its capabilities through plugins. In model training and research scenarios, hyperparameter tuning often faces problems such as large search spaces, messy experiment records, and difficulty preparing tables for papers.

quick-research-campaign is a DSH plugin for research pipelines. It does not directly run experiment training; instead, it focuses on planning the search space, recording experiment status, comparing baselines and SOTA, and generating materials that can be used directly in papers. The plugin converts hyperparameter search into a bounded, deconfounded trial matrix and automatically calculates marginal effects and confounding.

Plugin Information¶

• Name: quick-research-campaign
• Maintainer: wswbx
• License: MIT
• Focus: research pipelines, hyperparameter tuning, experiment archiving

Installation and Configuration¶

Installing the plugin requires running DSH plugin management commands. After installation, the corresponding profile must be restarted for changes to take effect.

Installation¶

dsh plugin --profile web add quick-research-campaign

If you need to install it for local development or in an unpublished environment, you can use a local file path:

dsh plugin --profile web add "file:C:/path/to/quick-research-campaign"

Uninstallation¶

dsh plugin --profile web remove quick-research-campaign

Configuration¶

The plugin injects configuration through cordis.patch.yml. Users can also override the entire config section.

- id: tool-research-campaign
  config:
    maxTrialsPerCampaign: 64

Core Features¶

The plugin provides three tools and one skill.

1. research_sweep_plan: Search Space Planning¶

Expands the search space into a bounded, deconfounded trial matrix.

• Grid enumeration and sampling: When the budget is sufficient, enumerate the full Cartesian product; when the budget is insufficient, use uniform sampling to avoid parameter resonance caused by fixed-step truncation (i.e., systematic correlation across adjacent dimensions).
• Detecting missing cells and confounding: Reports which parameter values were never tested and which parameter pairs are confounded (their values correspond one-to-one, making their contributions inseparable).

Parameter definition example:

space: { 
  lr: {min:1e-5, max:1e-3, steps:4, log:true}, 
  window: [4096, 8192, 32768], 
  gate_type: ["sigmoid","silu","none"] 
}

2. research_sweep_record: Experiment Bookkeeping¶

Records the status and metrics of each trial.

• Status recording: Records ok (successful), failed (failed), and pruned (pruned) statuses.
• Full recording: Failed and pruned runs must also be recorded; otherwise, the boundaries of the search space cannot be judged.
• Data validation: Runs with status ok but no metrics are rejected.

3. research_sweep_report: Result Reporting and Attribution¶

Ranks results by metrics and generates comparative reports.

• Comparative analysis: Compares baselines and SOTA and calculates relative gaps.
• Marginal effects: Calculates the marginal effect of each parameter and identifies paper-headline components.
• Confounding checks: Checks confounding among parameters.
• Materials checklist: Generates a paper materials checklist; unchecked items indicate missing content in the experiment section.

Report output example:

| # | trial | acc_32k | vs baseline | vs SOTA | gate_type | lr | window |
| 1 | t0011 | 72.7 | +4.6 WIN | +1.5 WIN | silu | 0.001 | 32768 |

- best over sota target: +1.5 (target beaten)

### gate_type
| value | mean acc_32k | trials |
| silu    | 71.625 | 4 |
| sigmoid | 70.7333 | 3 |
| none    | 68.9 | 3 |

4. Skill research-campaign: Pipeline Handbook¶

A skill containing an eight-stage pipeline handbook.

• Single component, single switch: Core design principle; component switches are off by default, and effects are validated by enabling them one by one.
• Progressive validation: First ensure a single trial works end-to-end, then scale to large-scale concurrency, avoiding wasted compute when the pipeline is not yet functional.

Usage Limits and Cautions¶

• Process-state ledger: The ledger is process state, indexed by the calling session, and disappears when the process exits. For multi-day searches, records need to be manually written to research/runs.jsonl to rebuild the ledger.
• Does not execute experiments: The plugin does not run training jobs; it only plans, records, and ranks. Therefore, the ledger can contain trials that never occurred.
• Report authenticity: The report checks completeness (whether required paper materials are present) but not truthfulness (it cannot detect metrics recorded from the wrong source).
• No scheduling: The plugin tools do not handle scheduling logic such as queueing, retries, or preemption. These belong to DSH’s job / schedule capabilities.
• Environment requirements: Requires Node.js version ^22.19.0 || >=24.0.0.
• No UI interactions: The plugin has no UI, does not send HTTP requests, and does not execute experiment tasks.

Summary¶

quick-research-campaign is a toolkit focused on research workflows. It addresses the chaos and uncertainty in hyperparameter tuning by providing automated matrix planning and attribution analysis, helping developers quickly produce paper-level data tables.

• GitHub repository: https://github.com/wswbx/quick-research-campaign
• Skill directory: https://www.skillhub.cn/plugins/wswbx/quick-research-campaign