Foreword

DeepSeek Harness (DSH) uses a plugin-based architecture. As the number of models grows, managing the actual availability (Alive/Dead) of each model becomes difficult. DSH Model Probe is a third-party plugin designed to integrate model health monitoring into the DSH Web UI, providing persistent status, runtime monitoring, and global sorting.

Plugin Positioning

DSH Model Probe is a persistent model health monitoring plugin for the DeepSeek Harness Web UI.
- Maintainer: AGSQ11
- License: MIT
- Core value: Ensures accurate model selection in DSH through runtime probing, automatic recovery, and history tracking.

Core Features

The plugin provides the following capabilities under Settings → Model Probe in the DSH Web UI:

  1. Native UI Integration
    Provides a dashboard in Settings to display the model list and their health status.

  2. Global Health Badges
    In the DSH model selection list, alive models display a green dot, while dead models display a red dot. Unprobed models have no dot.

  3. Health-First Sorting
    The model list is sorted in the following order:
    - Alive (green dot)
    - Unknown (no dot)
    - Dead (red dot)
    Within each tier, models are sorted alphabetically.

  4. Quick Filtering
    Supports case-insensitive prefix search. It works on both model display names and model IDs.

  5. Persistence and Monitoring
    - Health status is persisted on the host side and remains after browser and DSH restarts.
    - Runtime monitoring of normal DSH llm/stream calls.
    - History is limited to the most recent 500 results.

  6. Automatic Recovery
    Dead models are automatically re-probed every 30 minutes. If successful, they are restored to Alive.

  7. Manual Probing
    Provides a “Probe all” feature, with configurable timeout and concurrency.

Installation and Enablement

After cloning or downloading the repository, add the local plugin directory to the DSH configuration file:

dsh plugin --profile default add ./dsh-model-probe

After adding it, restart DSH and access Settings → Model Probe to use it.

Usage Instructions

  1. Access the Dashboard
    Open the DSH Web UI and navigate to Settings → Model Probe.

  2. Manual Probing
    Use the “Probe all” button and configure the probe timeout and concurrency.

  3. Status Path Override
    To change the storage location of the status file, set the environment variable:

DSH_MODEL_PROBE_STATE_FILE=/custom/path/state.json

Health Policy and Notes

The plugin uses specific health determination logic:

  • Probing Strategy: Manual probing makes up to 3 attempts. If all fail, the model is marked as Dead.
  • Runtime Failures: If a normal DSH request fails due to a terminal error, the plugin triggers two health retries. The model is marked as Dead only if both retries fail and there is no successful request recovery.
  • Minimal Probe Request: The plugin does not replay the user’s original request; instead, it sends a minimal request (Reply with exactly OK., maxTokens=8) to avoid duplicated output.
  • Cancellation Handling: Explicit user cancellations or aborts are not treated as model death.
  • Key Logic: Health status is keyed by the Provider + Model combination, not by model name alone.
  • Security Note: The plugin does not store any provider credentials, and probe requests use adapters already configured in DSH.
  • Directory Limitations: The DSH provider directory is for reference only. The plugin checks published models and cannot infer hidden model IDs that are not exposed by the directory.

Status Persistence

The plugin stores current health status and history in a state.json file.

Default Paths:
- Windows: %LOCALAPPDATA%\dsh-model-probe\state.json
- macOS: ~/Library/Application Support/dsh-model-probe/state.json
- Linux/Unix: $XDG_STATE_HOME/dsh-model-probe/state.json or ~/.local/state/dsh-model-probe/state.json

Updating

When updating, replace the plugin directory with the new version source, keep the entry in the DSH configuration file unchanged, and then restart DSH. Persistent state is stored independently from the plugin directory.

Conclusion

DSH Model Probe addresses the blind spot in model availability management by embedding health monitoring into the Web UI. It uses DSH’s own runtime for probing and strictly follows health-first sorting logic, making it suitable for developers who need a stable LLM channel.