dsh-cyberdog-speech-sherpa

Voice is the most natural interface for an AI assistant, yet most solutions lean on cloud APIs—sending audio off-device, incurring usage fees, and going silent when the network drops. dsh-cyberdog-speech-sherpa brings the entire speech pipeline on-premises with Sherpa-ONNX: streaming Chinese STT powered by Zipformer and multi-voice Chinese TTS powered by VITS, all fully offline, free, and running comfortably on a standard CPU.

The plugin exposes three core tools. stt_transcribe converts audio to text, tts_speak synthesizes text into multitone speech with adjustable rate and voice ID, and sherpa_models_download lets the agent self-heal by fetching missing models with resume support. On the WebUI side, a push-to-talk microphone button handles the full loop: press to record (up to 120 s with a countdown), release to stop, the browser encodes the recording to WAV, the transcript is auto-filled and submitted—mimicking the feel of a mainstream voice assistant.

It is well suited for privacy-conscious developers, offline or air-gapped deployments, and anyone who wants to add Chinese voice interaction to DeepSeek Harness with minimal hardware overhead. The plugin has zero runtime framework dependencies, total model size of roughly 156 MB, and is released under the MIT license, making it easy to drop into any existing workflow.