dsh-local-llm
Run the following command in DeepSeek Harness:
dsh plugin install wertyBSd/dsh-local-llm
Paste the following prompt into your AI chat to install this plugin:
Run dsh plugin install wertyBSd/dsh-local-llm in your terminal to install the plugin; the source repository is at https://github.com/wertyBSd/dsh-local-llm .
About this plugin
Running a large model locally usually means installing Ollama, accepting its version lock-in, and wrestling with opaque configuration. dsh-local-llm removes that middleman entirely: it pulls the official llama-server binary from llama.cpp releases, binds it to 127.0.0.1, and lets you grab GGUF models straight from Hugging Face or any direct URL. The whole inference stack is just DeepSeek Harness plus llama-server - transparent and minimal.
The sidebar exposes a Local models panel with a curated catalog (Mistral, Llama 3, DeepSeek Coder, Qwen 2.5) and a free-form field for any .gguf link. Downloads stream progress over SSE, deduplicate concurrent requests, clean up interrupted files, and guard against path-traversal in filenames. On the runtime side you can switch between CUDA, CPU, or automatic builds, and the choice persists across Harness restarts. Context size is auto-selected per model, and for small models like TinyLlama the adapter estimates prompt size and rejects oversized requests before they ever hit the wire, avoiding confusing server errors.
The plugin registers itself as the local-llm provider in Harness, converts tool calls to the OpenAI function-tool schema, and streams through /v1/chat/completions. The UI speaks ten languages, with the selection stored in the browser. This fits developers and researchers who want full control over their local inference runtime without the Ollama dependency, and who want large-model capability woven directly into a DeepSeek Harness workflow.
Use Cases
- Run 7B/8B models offline on a LAN for code completion and chat
- Pick a local LLM inside DeepSeek Harness to replace a cloud API call
- Toggle between CUDA and CPU builds for heterogeneous hardware deployment
Best For
- Users who want to skip Ollama and manage GGUF files directly
- Developers and researchers adding local LLMs into a DeepSeek Harness setup
- Engineers who need fine-grained control over context size and build selection
Related Plugins
ModLens is a vision plugin for DeepSeek Harness that gives text-only models sight by reading images pasted directly into chat, with zero-config setup and multiple vision engines.
On-demand vision for text-only DeepSeek Harness agents: built-in free keyless vision chain and 14 vision tools, routing image turns as tool calls to vision models with pixel fidelity, no Python needed, one-command install.
Give text-only models in DeepSeek Harness eyes, enabling image Q&A, long-screenshot OCR, UI restoration, and GUI visual tasks.
An armor-breaking plugin for DeepSeek: rewrites instruction priority, expands refusal-blocked phrases, persists across sessions, and shows a green active indicator.