dsh-llm-gate

Local LLM backends like llama-server often accept only one or two concurrent requests. When the main agent, a subagent, and a compaction job in DeepSeek Harness overlap, the surplus requests are silently parked by the server. The client cannot tell "waiting for a slot" from "dead", and the Node HTTP layer times out after 300 seconds with a terminated error, killing the entire turn. dsh-llm-gate intercepts those surplus requests inside dsh before any HTTP call is made: they sit in a FIFO queue with no timeout clock running, and are dispatched as soon as a slot frees up. You configure maxConcurrent, maxQueued, and queueTimeoutMs per provider, so an overfull queue fails fast with QUEUE_FULL or QUEUE_TIMEOUT instead of hanging forever. The plugin hooks into the llm/stream waterfall, so it covers every model request in the host—agents, subagents, compaction, and title generation—without any per-site code changes. It is built for developers who plug local or low-concurrency LLM providers into DeepSeek Harness and want to eliminate timeout cascades caused by concurrent overflow, without hand-rolling semaphores or retry loops in their own code.