dsh-context-budget

Running large models on local GPUs means the context window a model advertises is often far beyond what the hardware can actually process efficiently. A 27B model may claim 256K context, yet prefill speed can drop from 300 tokens/s to 70 tokens/s past 100K, and the GPU may hang before the window is even full. dsh built-in compaction triggers at a fixed ratio and is blind to real performance, so early turns return in seconds while later turns in the same session take half an hour or freeze the card outright.

dsh-context-budget runs three real-time checks before every agent step on each guarded provider route: a hard token ceiling (hardCeilingTokens), the last observed time-to-first-token (maxTtftMs), and a predicted cold-prefill wait derived from the measured prefill rate (maxColdPrefillMs). The prefill rate is sampled from dsh usage reports and wall-clock timing, so it works with any provider dsh streams from without touching the model server directly. Unset checks are simply skipped.

When a check trips, the plugin can warn or compact. A warn prints one diagnostic line to the dsh terminal naming the check, the current values, and an estimate of what compacting right now would cost. A compact calls the dsh compaction engine to summarize everything except the most recent retainTokens messages, cut at a tool-call boundary, and prints the outcome. Because a local cold compaction itself has a prefill cost, a lower ceiling with a larger retainTokens usually costs less overall than many small compactions.

This plugin is for anyone running local models through dsh who wants context compaction driven by measured performance rather than a fixed ratio. It works alongside dsh compaction-basic, which keeps its own static threshold. The plugin adds the dynamic measured checks and the compaction cost estimate on top, so both can coexist without conflict.