Introduction

The philosophy of DSH is “Everything is a plugin”. The community directory mentioned here is an independent site and does not have an official subordinate relationship with DeepSeek / Huanfang.

Regarding the DSH web client, besides text input, there is a specific requirement: seeing real-time subtitles while speaking, and the ability to be interrupted when the assistant replies. haoku123/dsh-voice provides exactly this voice mode.

What is This

haoku123/dsh-voice is maintained by haoku123 and is licensed under MIT. Its positioning is the full-duplex voice mode of DeepSeek Harness: streamed ASR → LLM → TTS with barge-in.

The browser side is responsible for recording and sending raw little-endian f32 PCM, while the host side runs ASR and TTS. The DSH web client injects:

  • @deepseek-ai/dsh-client-runtime
  • @deepseek-ai/dsh-client-ui-slots

Core Features

Below are a few points most relevant to actual usage.

Voice Pipeline

Recording first, then recognition, then submitting to the model, and finally reading out the response:

  1. The browser records audio to obtain raw little-endian f32 PCM.
  2. Recognition is handled by SenseVoice on the host side, running via sherpa-onnx; the output includes native simplified-Chinese output, punctuation, and inverse text normalization (ITN).
  3. The model response passes through the llm/stream tap. This tap is lossless: chunks yielded unchanged, and the sentence segmenter observes them.
  4. TTS uses msedge-tts, streaming back audio frames and caption text via SSE.

Three Speaking Modes

The plugin provides three dictation gestures:

Gesture Behavior
tap the mic continuous dictation; VAD segments on trailing silence
hold the send key（or the mic） records until release; slide up to discard
hold Ctrl（可配置 asr.hotkey） press-to-talk without leaving the keyboard; Esc discards

If using press-to-talk, it bypasses VAD; captures shorter than 250ms are treated as mis-taps and discarded.

Real-time Captions

When holding the microphone or send key, an overlay displays live caption: interim transcript preview, and keeps a spinner until the final transcript lands.

Here is a boundary: SenseVoice is not a streaming model. The interims for live captions are previews only; captures exceeding 12 seconds will stop, and if an interim lands after release, it will be discarded to avoid overwriting the final transcript.

Barge-in and Echo Cancellation

Barge-in is triggered by the leading speech edge. After triggering:

  • stops local playback;
  • drops queued and in-flight host TTS synthesis;
  • cancels the running turn.

Echo cancellation uses the NLMS acoustic echo canceller, using page TTS playback as the echo reference.

Endpoint Detection and Model Caching

Endpoint detection uses RMS endpoint detection: 16kHz getUserMedia, 2s trailing-silence cutoff, max 30s segment, with pre/post padding.

Model files are obtained via the cache-through HF model proxy, supporting resumption from partial .part files. You can run npm run prefetch for a warm-up.

modelHost accepts any HF-compatible mirror, for example:

https://hf-mirror.com

Installation and Enablement

First, confirm the environment meets requirements: Node ≥ 22.19 or ≥ 24 is required because the zstd APIs from node:zlib are used.

Then, enable the plugin with the install command:

dsh plugin --profile web add <repo-url-or-path>
dsh --profile web

The <repo-url-or-path> above is a placeholder and needs to be replaced with actual installation parameters provided by the repository.

Main configuration items relevant to this article:

asr:
  hotkey: Control
  modelHost: https://hf-mirror.com

Peer dependencies include:

  • @deepseek-ai/cordis ^4.0.1
  • @deepseek-ai/dsh-web ^0.1.0-rc.6
  • react ^18.2.0

Typical Usage

First, warm up the model cache:

npm run prefetch

This command defaults to:

http://127.0.0.1:3080

Then, choose the input method based on the scenario:

  1. tap the mic: continuous dictation, VAD segments on trailing silence.
  2. hold the send key or hold the mic: records until release; slide up to discard.
  3. hold Ctrl: press-to-talk without leaving the keyboard; press Esc to discard.

After the above steps, speech is recognized, submitted, and streamed back as audio replies with captions.

Use Cases and Notes

Suitable for using voice input in the DSH web client, especially scenarios requiring Chinese recognition output, real-time captions, streaming TTS, and the ability to interrupt the assistant during playback.

Notes for use:

  • SenseVoice is not a streaming model; the interims of live captions are just previews and cannot be considered the final transcript.
  • Live captioning stops after 12 seconds of audio; interims arriving after release are discarded.
  • Press-to-talk bypasses VAD; captures shorter than 250ms are discarded.
  • RMS endpoint detection has parameters for 16kHz, 2s trailing-silence cutoff, and max 30s segment.
  • The plugin requires Node ≥ 22.19 or ≥ 24.
  • The plugin runs with the current dsh process permissions; the source code and MIT license should be checked before installing.

Links

  • GitHub: https://github.com/haoku123/dsh-voice
  • Directory Page (source from plugin clue): https://www.skillhub.cn/plugins/haoku123/dsh-voice