dsh-local-models

Running local LLMs has always felt like glue-work: terminal commands, hand-wired OpenAI-compatible endpoints, guessing whether VRAM holds the model, and restarting servers every time you switch. dsh-local-models pulls the entire workflow into a single tab in the dsh Web GUI, so managing local inference feels as straightforward as configuring a cloud provider.

Open the tab and browse for a .gguf file right in the app. The plugin parses the GGUF header to surface architecture, quantization, layer count, and MoE detection without loading weights. From there you tune the context slider, MTP speculative-decoding depth, thinking level, and optional vision mmproj, with MoE expert-placement controls for mixed GPU/CPU layouts. A live VRAM estimate tallies weights, KV cache, recurrent state, and compute overhead against your target GPU, flagging fit, safe margin, and the maximum context that still fits. Save named profiles to swap configurations in one click, or launch router mode to serve every saved profile from a single OpenAI-compatible endpoint—models load on demand and auto-register in dsh the moment the router starts.

This plugin is for developers and power users who run stock upstream llama.cpp on a local GPU and want dsh to talk to their local models without extra glue. It depends on no forks, no build patches, and a zero-dependency client bundle, making it a drop-in addition to an existing llama.cpp setup.