AI Agent Hub
Back to plugins
🤖

dsh-llamacpp

Model Inference Updated 2026.09.01

Run the following command in DeepSeek Harness:

dsh plugin install jwilson411/dsh-llamacpp

Paste the following prompt into your AI chat to install this plugin:

Run dsh plugin install jwilson411/dsh-llamacpp in DeepSeek Harness to install this plugin; the source code lives at https://github.com/jwilson411/dsh-llamacpp .

About this plugin

You already run llama-server with your weights loaded, yet DeepSeek Harness ships no native provider that can reach it. dsh-llamacpp is the missing bridge: it bypasses Ollama's model registry, /api/* routes, and automatic model pulls entirely, and speaks directly to the OpenAI-compatible /v1/chat/completions endpoint on your local llama.cpp binary.

Once registered, llamacpp becomes a standard provider route on the LLM seam, and the harness dispatches model calls through ctx.llm.stream(). Each call is a single POST with stream: true; options.system is prepended as a system message, options.tools maps to the OpenAI tools array, and temperature, stop, and maxTokens pass through untouched. On the way back, the adapter emits a strict StreamChunk sequence—block-start, per-delta text chunks, block-end, usage, finish—so an empty response never leaves an open block. Any failure surfaces as an LlmError with a stable code (PROVIDER_UNREACHABLE, PROVIDER_HTTP_ERROR, PROVIDER_ERROR, PROVIDER_PROTOCOL_ERROR, UNSUPPORTED_CONTENT); nothing is silently swallowed into an empty stream.

It stays deliberately thin: text in, text out. Image, tool-call, and tool-result blocks are refused rather than dropped, because silently removing any one would hand the model a conversation the caller never intended. It registers nothing on ctx.tools and holds no opinion about -ngl, tensor split, or any offloading flag. If your inference stack is Ollama, this is the wrong plugin. It is built for local-inference users who compile or download llama.cpp themselves, manage context windows manually, and want a predictable, zero-magic, swappable provider route under the harness.

Use Cases

  • Run inference on a local llama-server without Ollama or cloud APIs
  • Hot-swap between multiple local models by changing the llama.cpp --alias flag
  • Wire a zero-magic, swappable local provider route into a Harness-driven chat application

Best For

  • Local-inference developers who compile or download llama.cpp and manage their own weights
  • Teams that want to run chat applications on a single machine with zero cloud dependency
  • Plugin developers extending provider routes under the DeepSeek Harness