Preface

DeepSeek Harness (DSH) adopts an “everything is a plugin” architecture. When optimizing model inference, especially during S1-stage (shadow mode) observation or S3-stage (canary release) rollout, the system must dynamically decide whether to use a small language model (SLM) or a large language model (LLM) based on the content of user utterances. The dsh-session-slm-router plugin calls the external vertical-small-model CLI tool to predict a weak/strong intent classification for each turn and records the result in a shadow log. In default mode, it only records data and does not change the active session model; under certain configurations, it can perform downgrade operations.

Core Features

  • Shadow-mode SLM routing: Predicts the weak/strong classification (weak/strong) of each user turn via the vertical-small-model CLI.
  • Shadow logging: Writes prediction results, actual model tier, downgrade decisions, and other data to a JSONL-format shadow log file.
  • S1 mode (default): Only predicts and writes logs; does not change the currently active model in the session.
  • S3 mode (weak-model mode): Performs downgrade model switching, but only allows switch_to_weak (downgrade); switch_to_strong (upgrade) is recorded but not allowed.
  • Reasoning Effort Compatibility Check: In weak-only mode, binding checks the target model’s reasoning effort capability. If unsupported, it strips the parameter or defers to the next choice.
  • Consent Gate: Supports confirming downgrades through a host approval channel (such as a browser UI popup) before execution, with support for always/smart/never policies.

Installation and Enabling

The plugin is installed through DSH’s plugin manager. Before installation, ensure the external dependency vertical-small-model CLI has been deployed.

# 安装插件
dsh plugin add dsh-session-slm-router

After installation, enable it in the configuration file and configure the relevant parameters.

Configuration

Mode Selection

The plugin provides three operating modes:

  • shadow (default): only predicts + writes shadow logs, does not switch models.
  • weak-only: actually switches models, but only allows switch_to_weak (downgrade); switch_to_strong is recorded but not allowed.
  • off: disables the plugin.

Required Configuration

At runtime, you must configure the prediction command and model path, pointing to the deployment location of the vertical-small-model project.

session-slm-router:
  mode: shadow  # 或 weak-only
  predictCmd: "python3 <path-to>/vertical-small-model/scripts/route_predict.py"
  predictModel: "<path-to>/vertical-small-model/data/eval/routing-v0/model-r1.json"
  logPath: "slm-shadow/session-slm-shadow.jsonl"

Optional Configuration Items

  • Consent Gate (downgradeConfirm): Defaults to never (silent automatic downgrade). Options: always (popup on every bind), smart (popup only in high-intent scenarios, such as the first conversation turn or current strong model ≠ default), never (no popup).
  • Slot Cache (slotCachePath): Optional. Used to persist the health status of weak model slots; the path is relative to ~/.dsh. Delete the cache file to rebuild it.
  • Slot Rules (tierRules):
    • maxTokensWeakMax: L2 metadata downgrade threshold.
    • autoWeak: Automatic inclusion mechanism, enabled by default. When new provider/model metadata is confirmed (maxTokens ≤ threshold), it is automatically added to the downgrade candidate pool, unless explicitly denied in strongSlots.

Full Configuration Example

session-slm-router:
  mode: weak-only
  weakOnlyMainOnly: true  # weak-only 是否只对主会话换模
  predictCmd: "python3 /path/to/vertical-small-model/scripts/route_predict.py"
  predictModel: "/path/to/vertical-small-model/data/eval/routing-v0/model-r1.json"
  logPath: "slm-shadow/session-slm-shadow.jsonl"
  slotCachePath: "slm-shadow/slot-order-cache.json"
  tierRules:
    maxTokensWeakMax: 32768
    autoWeak: true
  downgradeConfirm: never
  weakSlots:
    - { provider: openai, model: gpt-4o-mini }
  strongSlots:
    - { provider: openai, model: gpt-4o }

Data Recording

The shadow log file is located at ~/.dsh/slm-shadow/session-slm-shadow.jsonl, with one JSON object per line. Key fields include:

  • suggested_tier: weak/strong classification predicted by the CLI.
  • actual_tier: current actual model tier.
  • target_source: downgrade target source (registry/strongSlots or auto).
  • switch: decision result (stay / switch_to_weak / switch_to_strong).
  • bound: whether the model was actually switched in weak-only mode.
  • downgrade_confirm: consent gate result (confirmed/denied/cancelled/unavailable).
  • predict_ok: whether prediction succeeded.

Notes and Applicable Scenarios

  • External deployment dependency: The plugin itself does not include the prediction model. You must deploy the vertical-small-model CLI and correctly configure predictCmd and predictModel.
  • Permissions and runtime environment: The plugin runs with DSH process permissions. Ensure the configured paths are writable.
  • Positioning: The DSH community directory is an independent site and has no official affiliation with DeepSeek / High-Flyer. This plugin aims to provide flexible shadow mode and canary routing capabilities.

Conclusion

This plugin provides DSH users with a practical shadow-mode routing solution, making it easy to validate routing strategies during the S1 stage or perform canary releases during the S3 stage. Through JSONL logs, each round’s prediction and decision process can be clearly tracked.

  • GitHub Repository: https://github.com/NinjaSln-labs/dsh-session-slm-router
  • Community Directory: https://www.skillhub.cn/plugins/NinjaSln-labs/dsh-session-slm-router