dsh-plugin-local-model

Running a local GGUF model inside DeepSeek Harness used to mean manually spawning llama-server, babysitting VRAM, and retyping a wall of flags every time you wanted different sampling. This plugin folds all of that into a single settings panel: the very first message auto-loads the model via llama.cpp, and after five minutes of silence it unloads cleanly and frees the GPU. No lifecycle scripting on your side.

On the parameter front, the plugin surfaces more than a dozen settings including KV cache strategy, the full sampling set, vision projection, Multi-Token speculative decoding, and reasoning level, each with its default, a human-readable note, and a one-click restore. The named parameter presets pinned at the top of the panel let you save a whole parameter bundle under a label and switch between them instantly, while environment-level items like ports, directories, and keys stay untouched. When VRAM is tight, the default auto GPU-layer mode lets llama.cpp fit layers to available memory instead of forcing an OOM.

It suits developers who want a local LLM as a daily driver inside dsh without leaving their workflow: you place your own llama binaries and GGUF files into the designated folders, and the plugin never fetches models over the network, never reads workspace files, and never listens on anything but the local loopback. Whether you are A/B-testing quants, dialing in sampling, or toggling reasoning on a thinking model, it is a few clicks in the panel, no terminal required.