Introduction

The DeepSeek Harness (DSH) official host does not include speech-processing capabilities (it supports only LLM chat modality and image attachments). When developing agents that require speech input, output, or real-time transcription, developers often need to integrate ASR (Automatic Speech Recognition) and TTS (Text-to-Speech) services themselves. The dsh-speech plugin registers independent HTTP/WS speech service routes through DSH extension points, providing token-gated speech transcription and synthesis capabilities to the host without modifying DSH source code.

What It Is

dsh-speech is a speech capability plugin for DeepSeek Harness, maintained by agent-mobile under the MIT license. It exposes audio transcription (ASR) and synthesis (TTS) services to clients through the /s/api and /s/ws route families, supporting multiple cloud providers and local services.

Core Features

  • Multi-provider support: Supports four adapter types: dashscope, openai-compatible, local-relay, and streaming-ws.
  • Alibaba Cloud Bailian native protocol: Supports batch recognition/synthesis and sentence-level real-time transcription.
  • Generic cloud provider support: Compatible with OpenAI, Groq, SiliconFlow, Fireworks, and similar APIs.
  • Direct local service integration: Supports SenseVoice and CosyVoice2.
  • Real-time transcription: Supports cloud streaming and local VAD (Voice Activity Detection).
  • Cloud synthesis: Supports iFlytek TTS.
  • Flexible mix-and-match: ASR, TTS, and real-time transcription selectors can be configured independently.
  • Model management: Includes a built-in button to fetch the latest platform models.
  • Credential security: Supports inline entry or environment variable references.

Installation and Enablement

Use the official installation command to add the plugin. During first installation, pnpm may ask whether to allow building this package; please follow the prompt to proceed.

dsh plugin --profile web add github:agent-mobile/dsh-speech

After installation, start dsh web; configuration items take effect automatically when the host starts.

Typical Use Cases

After installing the plugin and starting the host, you can configure the following:
1. Configure cloud providers such as Groq and SiliconFlow as ASR/TTS backends.
2. Configure local SenseVoice/CosyVoice2 services.
3. Configure Deepgram streaming real-time transcription.

Use Cases and Considerations

  • Use cases: Private on-premises voice API deployment, switching among multiple cloud providers, and local offline speech processing.
  • Considerations:
    • The desktop entry plugin includes a browser pane; you can hide this tab by setting uiEntry: false.
    • Port binding is handled by mobile-gateway (bound to 0.0.0.0).
    • Token is required; other configuration items have default values.
    • Bailian real-time transcription does not support speaker diarization (available only via local-relay).
    • The plugin runs with the permissions of the current dsh process; please review the source code and license.