dsh-llm-auto-route: Auto Routing and Failover Plugin for DeepSeek Harness

Introduction¶

DeepSeek Harness (DSH) emphasizes the “everything is a plugin” architecture. When building agents that use multiple model inference providers, developers usually need to explicitly specify the provider and manually switch when a service fails. This increases configuration complexity. The dsh-llm-auto-route plugin aims to solve this problem. It simplifies multi-provider management by automatically discovering configured providers, matching request rules, performing health checks, and executing failover before output.

Plugin Overview¶

dsh-llm-auto-route is a community-maintained Cordis plugin developed by user qinyu765. It is positioned as a routing policy layer, responsible for deciding which dsh-llm-pi-ai route to use before a request is sent and for executing failover when necessary.

It does not implement the HTTP protocol, nor does it provide a provider SDK or register adapter routes (such as openai or deepseek). These functions remain the responsibility of the official @deepseek-ai/dsh-llm-pi-ai plugin.

Core Capabilities¶

The plugin primarily provides the following capabilities:

1. Provider Discovery and Matching

   • Automatic selection is triggered by setting provider: auto in the request options or by configuring provider as auto in the configuration file.
   • Matching uses a fixed precedence order: explicit (explicitly specified) → provider_env (environment variable detection) → base_url (base URL) → model_prefix (model prefix).
   • Environment variables (such as DEEPSEEK_API_KEY) and specific ports (11434 for Ollama, 8000 for vLLM) are used as matching signals.

2. Health Checks

   • Three modes are supported: adaptive (reuses the official model discovery interface, with caching), off (skips pre-checks), and probe (probes on every request).
   • An in-memory cache and bounded timeouts are used to control performance.

3. Failover Before Output

   • Provider switching is allowed only before text, reasoning, tool calls, or chunk output begins.
   • A conservative policy is followed: aborted requests, requests with an explicit provider, or requests that have already started output are not retried.

4. Immutability

   • The plugin returns a new immutable request configuration and does not mutate the passed-in frozen request object.

Installation and Enablement¶

The plugin is installed via npm and requires Node.js >= 22.19.0 and DeepSeek Harness >= 0.1.0-rc.5.

pnpm add dsh-llm-auto-route

After installation, you need to load the plugin’s bundled cordis.patch.yml file. If developers are manually composing plugins, they must ensure that the official @deepseek-ai/dsh-llm-pi-ai adapter is loaded correctly before dsh-llm-auto-route.

Typical Usage¶

Configure Routing Policy¶

The plugin provides default rules, and developers can override the default behavior through YAML configuration.

provider: auto
precedence:
  - explicit
  - provider_env
  - base_url
  - model_prefix
healthCheck:
  mode: adaptive
  timeoutMs: 3000
  cacheTtlMs: 30000
failover:
  enabled: true
  maxAttempts: 3
diagnostics: info
routes:
  deepseek:
    apiKeyEnv: DEEPSEEK_API_KEY
    defaultModel: deepseek-chat
    priority: 10
  openai-compatible:
    apiKeyEnv: GATEWAY_API_KEY
    baseURLEnv: LLM_BASE_URL
    modelPrefixes: [gateway-]

Routing Field Descriptions¶

The routing fields in the configuration are hints only and do not directly configure an adapter:

Use the API in Code¶

The plugin exports the pure functions resolveRoute and normalizeConfig, which can be used to manually resolve routing decisions.

import { normalizeConfig, resolveRoute } from 'dsh-llm-auto-route'

const decision = resolveRoute(normalizeConfig(), {
  model: 'deepseek-chat',
  env: { DEEPSEEK_API_KEY: 'present' },
  registeredProviders: new Set(['deepseek']),
})

if (decision.kind === 'matched') {
  console.log(decision.candidate.provider, decision.candidate.model, decision.stage)
}

Use Cases and Notes¶

• Use Cases: Developers who need to manage multiple OpenAI-compatible or DeepSeek-specific providers within a single service; scenarios that require automatic routing based on environment variables or model names.
• Notes:
  • This is a community project and is not officially maintained by DeepSeek.
  • Ensure that the dependent @deepseek-ai/dsh-llm-pi-ai has route names configured correctly; otherwise, NO_REGISTERED_ROUTE may be reported.
  • Failover only applies to the pre-output stage and does not modify a frozen request object.
  • Review the source code to confirm that the license (MIT) meets your project requirements.

Conclusion¶

dsh-llm-auto-route provides a lightweight routing policy layer for the DeepSeek Harness ecosystem. Through automatic provider selection and failover before output, it reduces the maintenance cost of multi-provider deployments, making it suitable for high-availability agent development scenarios.

• Plugin directory page
• GitHub repository