Introduction

When using voice input in the chat box of DSH Desktop, dsh-audio-copilot provides a microphone button: click to speak, and after multi-engine transcription, it automatically fills into the input box.

What is this

dsh-audio-copilot is a DSH voice input plugin maintained by ai-yucheng. Its main capability is ASR/Voice Input: microphone recording on the browser side, calling the selected engine for transcription on the server side, and then filling the result into the chat input box.

The plugin documentation indicates it is based on the DSH 0.1.0-rc.7 line and the license is MIT. Prerequisites include DSH Desktop, Node.js ≥ 20, and ffmpeg / ffprobe in the PATH.

Core Features

  • Provides a microphone button next to the chat input box: click to speak, and transcription results automatically fill into the input box.
  • Supports four voice transcription engines: zhipu, local, gemini, and openai.
  • The local engine uses local faster-whisper offline transcription. Models are downloaded on first use, approximately 460MB for small or about 1.5GB for medium, making it available offline thereafter.
  • The zhipu engine uses GLM-ASR-2512, supporting Mandarin Chinese, Sichuan/Guangdong/Fujian/Wu dialects, and dozens of other foreign languages.
  • Maximum recording duration is 28 seconds; it stops automatically when the time is up. Longer audio requires segmented recording.
  • When the gemini engine encounters a 429 rate limit (free quota limit), it waits as prompted and retries once. The free tier allows 20 requests per period.
  • When the zhipu engine encounters webm, the server automatically converts it to 16kHz wav using ffmpeg.
  • If transcription fails, a prompt is given; if text injection into the input box fails, the result is automatically copied to the clipboard.
  • Configuration options have schema validation.
  • The maximum transcription file size is 25MB, with maxAudioBytes defaulting to 26214400; ASR timeout defaults to 120000 milliseconds.

Installation and Activation

First, confirm prerequisites: DSH Desktop is available, Node.js ≥ 20, and ffmpeg / ffprobe are in the PATH.

  1. Clone the repository in the DSH profile web directory:
git clone https://github.com/ai-yucheng/dsh-audio-copilot.git
  1. Add the plugin to the profile via a local link. Replace the path with the actual cloned directory:
pnpm add dsh-audio-copilot@link:C:/绝对路径/dsh-audio-copilot
  1. Append "dsh-audio-copilot" to the dsh.profile.bundles array in the profile’s package.json:
"dsh-audio-copilot"

Typical Usage

Configure API keys according to the selected engine. The documentation provides an example: configure ZHIPU_API_KEY when using zhipu, and GEMINI_API_KEY when using gemini.

Configure the plugin key in the profile’s cordis.patch.yml. The example below shows the keys that need to be configured; specific endpoints and model values should be filled in according to the selected engine:

- id: audio-copilot
  config:
    asrEngine: zhipu
    asrBaseUrl: <按所选端点填写>
    asrModel: <按所选模型填写>
    asrApiKeyEnv: ZHIPU_API_KEY

After configuration, execute the following steps in order:

  1. Restart DSH Desktop.
  2. Hard refresh the browser: Ctrl+Shift+R.
  3. Click the microphone button in the chat box toolbar and allow browser microphone permissions.
  4. Speak and stop; the transcribed text automatically fills into the input box.

You can verify the configuration using the following command:

dsh --profile web --dump-config | grep audio-copilot

Engine Descriptions

  • zhipu: Supports Mandarin Chinese, Sichuan/Guangdong/Fujian/Wu dialects, and dozens of other foreign languages; pay-per-use billing, insufficient balance will cause transcription failure.
  • local: Local faster-whisper offline transcription; models need to be downloaded for first use; dialect performance is average, it is recommended to switch to the zhipu engine for Cantonese.
  • gemini: The free tier allows 20 requests per period; automatically waits as prompted and retries once when encountering a 429 rate limit.
  • openai: Available as one of the optional transcription engines; configure the corresponding API key according to the selected engine.

Suitable Scenarios and Notes

Suitable for scenarios where voice input is used in the DSH chat box and the transcription result is expected to go directly into the input box. If focusing on Chinese dialects, prioritize zhipu; if offline capability or reducing external dependencies is needed, prioritize local; if using gemini or openai, prepare the corresponding API key first.

Since the plugin runs with the permissions of the current DSH process, and involves local recording, local ffmpeg / ffprobe, some external ASR services, and configuration validation, you should check the source code and license before installation. The license introduced in this article is MIT.

Links

  • Directory Page: https://www.skillhub.cn/plugins/ai-yucheng/dsh-audio-copilot
  • GitHub: https://github.com/ai-yucheng/dsh-audio-copilot