dsh-llama-model-manager

Running local llama.cpp GGUF models inside DSH works fine until you want to switch to a different one. Stop the server, edit the provider config, restart, and hope no request was mid-flight. On Windows the situation gets worse: Node's child.kill(SIGINT) silently compiles down to TerminateProcess, so llama.cpp never gets a chance to free its model and you lose a dozen gigabytes of VRAM every single time. dsh-llama-model-manager absorbs the entire lifecycle-start, stop, switch, recover-behind one stable OpenAI-compatible gateway, so DSH always talks to a single fixed URL while the model underneath changes freely.

Model switching is guarded by a serialization gate: in-flight inference holds a shared ticket, a switch request acquires an exclusive one and waits for every outstanding request to drain before the connection is torn down. Stopping a model delivers a genuine Ctrl+C console control event through a hidden PowerShell helper that performs the P/Invoke dance, letting llama.cpp run its own cleanup and call llama_model_free. Measured end-to-end on a 27B model: clean exit code zero, full VRAM returned, no window ever flashes. If DSH crashes mid-session, the plugin cross-checks the recorded pid, executable name, bound port, and reported model path before it will touch a leftover process, and it refuses to act on any pid it cannot positively attribute.

Built for Windows users who run local inference through DSH and switch between two or three GGUF models on a regular basis. It never reads or writes DSH's settings.yaml, ships zero npm dependencies and no build step, and exposes a live settings page where you can see the current model, tail logs, and start, stop, switch, or restart with one click. All you need is a llama-server.exe build and your .gguf files.