Introduction

In DeepSeek Harness (DSH) agent development, failures at the model or platform level (such as timeouts, rate limiting, or service unavailability) can cause agent requests to fail. Without an automatic failover mechanism, failures will continue to accumulate. The dsh-model-failover plugin uses a two-level circuit breaker strategy to automatically switch to a backup route when a model or platform failure occurs, ensuring service continuity.

Plugin Introduction

This plugin provides DeepSeek Harness with two-level model circuit breaking and failover capabilities.
* Maintainer: Letter2025
* Category: Model Inference
* License: MIT

Core Features

  • Two-Level Circuit Breaking: Supports model-level and platform-level circuit breaking. A model circuit is triggered when modelCircuitThreshold failures occur within the burstWindowMs window; a platform-level circuit is triggered when platformCircuitThreshold independent models are circuit-broken.
  • Failover: After the primary route is circuit-broken, the next request is automatically routed to the first healthy backup route configured in fallbacks.
  • Recovery Probing: After a circuit is opened, a recovery probe is performed after the modelCooldownMs cooldown period. If the probe succeeds, the circuit is closed; if it fails, the cooldown is extended.
  • Coordination with llm-retry: The plugin works together with the llm-retry plugin. llm-retry is responsible for request retries, while this plugin observes failures that escape retries, avoiding false circuit triggering caused by transient jitter.

Installation

Install the plugin via the DSH CLI:

dsh plugin --profile web add dsh-model-failover

Configuration and Usage

After installation, you need to configure fallbacks in cordis.patch.yml (this is a required field).

# cordis.patch.yml example
fallbacks:
  - provider: "provider-name"
    model: "model-name"

Additionally, the plugin automatically installs a companion skill configure-model-failover. This skill can guide you through setting up fallback model configurations via conversation, without requiring manual file copying.

Notes

  • Process-Local State: Circuit state is stored in memory and is reset when the plugin is reloaded. It does not support cross-instance sharing.
  • Only Routes agent-loop Calls: The plugin only routes the main-loop calls of agent-loop. Auxiliary calls such as session-title do not pass through the circuit logic.
  • No Platform Probing: The plugin only performs recovery probing for model-level circuits. Platform-level circuits recover only through the cooldown period.
  • Context Window Risk: If the context window of a fallback model is smaller than that of the primary model, switching may trigger a CONTEXT_WINDOW_EXCEEDED error. You need to configure stripReasoningEffort: true and choose a compatible fallback model.
  • Specific Retry Mode: When retryPolicy.mode is always, the plugin remains idle (by design), because the retry policy handles all failures in this case.

Summary

The dsh-model-failover plugin provides DeepSeek Harness with basic fault tolerance through a two-level circuit breaker mechanism and failover strategy. It does not modify core code and intervenes only in the form of a plugin, making it suitable for scenarios that require high-availability guarantees at the model layer.