dsh-tool-cassette

Agent workflow testing is plagued by two kinds of instability: model outputs that are hard to reproduce, and external tools that depend on live networks, third-party services, or a local process that may be down at any moment. While the official dsh-llm-replay plugin handles model-stream replay, tool bodies for HTTP, MCP, databases, and Python/Go/Java sidecars still make real requests, mutate data, and rely on services that are not always available. dsh-tool-cassette targets exactly that boundary: selected leaf tools execute for real the first time, and after that the same scene plays back completely offline.

Three core capabilities drive the plugin. First, recording: Record mode lets the tool body run exactly once, and the normalized value, error, and additionalContexts captured by DSH are written frame-by-frame into a versioned NDJSON cassette secured with a SHA-256 hash chain. Second, replay: Replay mode hits the exact call using structural path, lossless JSON parameters, and admission ordinal; the tool body invocation count is strictly zero, while the stored result still passes through the current output schema, renderer, and post-execution policy. Third, fail-closed safety: any trajectory deviation immediately poisons the replacer, preventing a mixed state where some hits come from the cassette and others hit production services, and the process exits with a non-zero code.

It is built for teams that need offline regression testing for Agent workflows; engineers who want stable CI reproduction of expensive or flaky tool responses; maintainers verifying that old results remain compatible after a schema or plugin upgrade; and developers debugging multi-language sidecar tools without repeatedly poking live services. Paired with the official dsh-llm-replay, it enables a fully keyless test pipeline where both the model stream and the tool stream are deterministic replays.