AI Agent Hub
Back to plugins
🤖

dsh-llm-auto-route

Model Inference Updated 2026.08.25

Run the following command in DeepSeek Harness:

dsh plugin install qinyu765/dsh-llm-auto-route

Paste the following prompt into your AI chat to install this plugin:

Install dsh-llm-auto-route in DeepSeek Harness by running dsh plugin install qinyu765/dsh-llm-auto-route; the source repository is https://github.com/qinyu765/dsh-llm-auto-route .

About this plugin

When your DeepSeek Harness setup carries multiple LLM routes—DeepSeek, OpenAI, Anthropic, Ollama, vLLM, or any OpenAI-compatible endpoint—hand-picking the provider on every request is tedious and error-prone. dsh-llm-auto-route sits as a pure routing-policy layer: it auto-discovers the best-fit registered route using API-key environment variables, LLM_BASE_URL port hints, and model-prefix signals, then explains its decision without ever leaking credentials.

Its failover logic is deliberately conservative. A route switch is allowed only before any visible output—text, reasoning, tool-call, or block—has been emitted. Explicit providers, cancelled requests, configuration errors, and already-streamed responses are never retried, so a single conversation can never be stitched together from two different models.

If you want to stop typing provider on every call in a multi-provider environment while still mixing local gateways (Ollama on 11434, vLLM on 8000) with cloud APIs, this plugin is built for you. It implements no HTTP protocol, ships no provider SDK, and registers no adapter routes of its own; it simply selects and explains a choice within the route directory already owned by the official dsh-llm-pi-ai, keeping every responsibility boundary intact.

Use Cases

  • Auto-selecting the best-fit LLM route across multiple configured providers with explainable decisions
  • A unified routing entry point for mixed local (Ollama/vLLM) and cloud-API dispatch
  • Conservative failover only before first visible output, preventing spliced multi-model responses

Best For

  • DeepSeek Harness users running multiple configured LLM routes
  • Developers deploying AI apps with mixed local-gateway and cloud-API backends
  • Engineering teams that want to stop specifying a provider on every single request