Introduction

The core design philosophy of DeepSeek Harness (DSH) is pluggability. The dsh-live2d-voice plugin adds a Live2D tab to DSH’s conversation view. It combines Live2D characters with the chat stream, provides real-time voice interaction, and solves the lack of dynamic visual expression in plain-text conversations.

Installation and Configuration

Dependencies and Installation

This plugin depends on a DSH environment version. Run the following command in the DSH profile directory (for example, ~/.dsh/profiles/web) to install it:

pnpm add dsh-live2d-voice

After installation, ensure that dsh.profile.bundles in the corresponding profile’s package.json includes the plugin name, then restart the DSH service.

Configuration File

The plugin uses the configuration file <DSH_HOME>/live2d-voice.json (defaults to ~/.dsh/live2d-voice.json). Save the configuration and refresh the page to apply changes.

{
  // 角色模型目录，包含 .model3.json 的文件夹
  "modelPath": "/root/.dsh/live2d-voice-models/haru",

  // Fish Audio 音色 ID
  "voiceId": "0c7771ca5910484e8a4933068017fcee",

  // Fish Audio API Key 列表（支持多 Key 轮询）
  "apiKeys": [],

  // 情绪标签映射到模型表情索引
  "emotionMap": {
    "neutral": 0, "joy": 1, "sadness": 2, "anger": 3,
    "surprise": 4, "fear": 5, "disgust": 6, "shy": 7
  },

  // 语音识别语言：auto / zh / ja / en
  "sttLanguage": "auto",

  // 角色说话语言：ja / zh / en
  "speechLanguage": "ja",

  // 字幕翻译目标语言：zh / ja / en / off
  "subtitleLanguage": "zh",

  // per-workspace 覆盖配置
  "workspaces": {}
}

Core Features

Live2D Rendering and Synchronization

The plugin is based on Cubism 4 models and renders with pixi.js v7. While speaking, the character drives ParamMouthOpenY according to the audio RMS envelope to achieve lip sync, and switches expressions automatically based on emotion tags.

Streaming Voice and Input

  • Streaming Audio: LLM responses are synthesized in streaming mode by Fish Audio, using 44100 Hz PCM, pushed via SSE, and played through a queue in the browser.
  • Continuous Voice Input: Supports browser-local VAD sentence segmentation, paired with real-time streaming to Volcano Engine ASR. Supports 25 languages and includes barge-in (interruption) support.
  • Voice Library: Built-in Fish Audio public voice library search, supporting keyword search and preview; selected voices are automatically written to the configuration.

Subtitles and Translation

Supports bilingual subtitle display for user and AI lines, fades out after 14 seconds, and can be hidden with one click. Character lines are automatically translated to the target language (requires setting subtitleLanguage).

Model Management

  • Multi-Model Directory: Since v0.3.0, a switcher appears in the settings panel when modelPath points to a directory containing multiple character subdirectories.
  • Per-Workspace Overrides: Since v0.3.0, voice, model, language, and other settings can be overridden by workspace path, enabling multiple workspaces with multiple characters.

Immersive and Experimental Features

  • Immersive Fullscreen: Since v1.1.0, one-click fullscreen is supported, and Wake Lock is enabled while listening to voice to keep the screen on.
  • Gaze Tracking: Experimental since v1.1.0, tracks gaze using a front-facing camera.
  • Gyroscope Parallax: Experimental since v1.2.0, adjusts the character’s head and body position based on device tilt angle.

Standalone Entry Points

  • Standalone Chat Page: Since v1.3.0, a standalone entry point without the GUI is provided, accessible at /live2d-voice/app?session=<id>.
  • Session Selection Page: Since v1.6.0, the /live2d-voice path is provided, listing all sessions and supporting quick creation.

Usage

Basic Conversation

  1. Open any session and switch to Live2D in the top view tab.
  2. Use the ⌨️ icon in the HUD panel to open the input box for direct chat, or use the 🎙️ icon to enable continuous voice input.
  3. When the AI is speaking, simply start talking to interrupt it (barge-in).

Using the Voice Library

In the Live2D settings panel or under System Settings » “Live2D Character & Voice”, click the voice row:

  1. Browse the trending list or use the search box to search the full library.
  2. Click to preview (pre-rendered samples are played when available).
  3. Click to select; the voice ID is automatically written to the configuration file.

Configuration Overrides

If you need to use different characters in specific workspaces, add the absolute workspace path to the workspaces field in the configuration file and specify the corresponding voiceId, modelPath, and other parameters inside the object.

Notes

  • Prompt Injection: The prompts injected by the plugin (such as voice output format and emotion tags) are session-scoped and take effect only when the current session has the Live2D view open. After switching back to normal chat, the model returns to standard output.
  • Network Dependencies: The TTS voice library and ASR recognition depend on the host-side network environments of Fish Audio and Volcano Engine; ensure that the API keys are configured correctly and the networks are reachable.
  • Touch Mode: The touch mode feature is currently pending release and is not yet available.