Introduction

In the DSH plugin workflow, model rollout is a common but inconvenient problem to manually verify: service providers may release new conversation models gradually, and determining which one the same account is actually assigned to often requires checking specific session outputs.

dsh-plugin-rollout-scout turns this into a batch of temporary probe sessions: it uses the current account to initiate probes, reads reasoning-delta during the chain-of-thought streaming output, performs heuristic scoring based on the characteristics at the beginning of each paragraph, and aborts the probe as soon as it is determined to be an old model.

What is this

dsh-plugin-rollout-scout is a DSH plugin maintained by SpookySandwich with an MIT license.

It is used to detect the conversation model currently assigned to the DeepSeek Harness account: it cancels the probe when judged as an old model, and allows it to run to completion and preserves the session when judged as a grayscale/new model.

The plugin explicitly states that it is a heuristic tool based on wording and is not an authoritative criterion.

Core Features

  • Uses the current account to start a batch of temporary probe sessions to detect the conversation model assigned to the account.
  • Reads reasoning-delta during the chain-of-thought streaming output and performs heuristic scoring based on the characteristics at the beginning of each paragraph.
  • Aborts the probe when judged as an old model; allows it to run to completion and preserves the session when judged as a grayscale/new model.
  • Supports setting probe prompts, models, concurrency, storage directory, and scoring threshold.
  • Supports early stopping switches such as the upper limit of TPS (tokens per second) generation speed and the lower limit of TTFT (time to first token) latency.
  • Provides a sidebar entry and a full-screen console, supporting Start, Pause, Resume, Force Stop, Clear Completed, and Delete All Sessions.
  • Probe sessions are automatically named, supporting hover-to-pin, retain, rename, and deleting old model sessions from disk.
  • The host provides a local /rollout-scout interface and requires write operations to include application/json and rejects cross-origin Origin.
  • Uses ownership records generated by the plugin to manage owned, protected, and deleting states to avoid mistakenly deleting non-probe sessions that are not retained.

Installation and Enablement

  1. Install via npm package:
dsh plugin --profile web add dsh-plugin-rollout-scout

  1. Restart DSH after installation.

  2. Alternatively, install directly from the repository:

dsh plugin --profile web add github:SpookySandwich/dsh-plugin-rollout-scout

web is the profile name and needs to be replaced with the actual profile used. The standalone version launches web, while DSH Desktop uses desktop.

The console entry is registered at the sidebar.footer.action slot. The DSH sidebar needs to declare this slot; otherwise, the console will have no entry point.

Typical Usage

  1. Open the console:

Open the “Grayscale Scout” console at the bottom of the sidebar, next to “Settings”.

  1. Set probe parameters:

In the left column, set the probe prompt, model (default V4-Pro / High), concurrency, storage directory, and scoring threshold.

  1. Enable early stopping and cleanup switches:

You can enable switches for the TPS generation speed upper limit, TTFT first token latency lower limit, automatic pause on strong match, discard Chinese chain-of-thought, and delete old model sessions.

  1. Initiate probe:

Click “Start”. During operation, you can “Pause”, “Resume”, or “Force Stop”.

  1. Retain sessions that need to be kept:

Hovering over a row will pin that row and reveal the “Retain” button. Clicking “Retain” keeps that probe session.

Retained sessions can be renamed.

  1. Clean up sessions:

You can “Clear Completed” to remove finished probes, or “Delete All Sessions” to clean up probe sessions in that directory.

If there are still sessions streaming output or releasing resources, use “Force Stop” before deleting to avoid affecting session files that are still being written.

  1. Check the storage directory:

The storage directory cannot be the home directory, disk root directory, or located within ~/.dsh.

  1. Run repository tests:
npm test

It covers tests for the classifier, interface protection, initiation failure behavior, and session deletion rules.

Applicable Scenarios and Notes

Suitable for developers who want to observe the conversation model currently assigned to their account in DSH and quickly abort sessions suspected of being old models.

It is suitable for grayscale scouting in personal workflows and is not suitable as an authoritative criterion for model versions.

Notes:

  • Each probe is a real conversation round and consumes its own quota; running tasks will continuously initiate new probes.
  • Concurrency, scoring threshold, TPS upper limit, and TTFT lower limit need to be adjusted according to the actual environment.
  • “Retain” does not make a round that is consuming tokens unstopable; “Force Stop” will still end that round, but the session and retain mark will be preserved.
  • Physical deletion only applies to sessions generated and recorded by the plugin itself; DSH blank workspace placeholder sessions and normal sessions manually created in the same directory will not be inferred as probes.
  • Write operations on the local /rollout-scout interface must include application/json content type, and cross-origin Origin is strictly rejected.
  • The plugin runs with the permissions of the current dsh process; you should check the source code, dependencies, and MIT license before installation.

Conclusion

dsh-plugin-rollout-scout organizes “starting temporary sessions, viewing chain-of-thought, judging models, and cleaning up old sessions” into a local plugin workflow that can be paused, retained, and deleted.

Repository: dsh-plugin-rollout-scout