Introduction

The design philosophy of DeepSeek Harness (DSH) is “everything is a plugin.” When developing intelligent agents with digital human capabilities, developers usually need to separately integrate ASR (Automatic Speech Recognition), TTS (Text-to-Speech), Avatar media rendering, and the dialogue pipeline. Finally, they also need to complete role registration and scene setup at the UI level. This process involves coordination among multiple components and has a relatively high development cost.

Live Talk is a plugin for DSH that provides a complete digital human clip solution. It combines the core of conversation character visualization, speech/video Provider interfaces, and the plugin shell. Developers can use it directly to drive a virtual character that naturally talks, turns around, and smiles in response to the conversation.

Plugin Overview

Live Talk is maintained by developer eddiehuang227-source and follows the MIT License. It allows users to transform any photo into a responsive virtual girl. The plugin performs well in low-latency, high-detail scenarios and provides a complete capability chain from ASR to TTS and then to Avatar rendering.

Core Features

Live Talk adopts a three-layer architecture, where each layer handles different responsibilities:

  1. DSH plugin shell: Located in package.json, cordis.patch.yml, src/index.js, and src/client.js. It is responsible for managing the DSH bundle/client manifest, Host route registration, rendering the browser conversation.view tab (character stage, opening video, and emotion clip synchronization), and keyless TTS preview.
  2. Conversation character visualization core: Located in src/core/characters.js, character-registry.js, dialogue-pipeline.js, emotion.js, and conversation-bridge.js. It is responsible for managing the character manifest, character registry, the provider-agnostic semantic pipeline (sentence splitting, emotion/action, TTS sanitization), and bridging DSH session events.
  3. Speech/video provider seam: Located in src/core/seams.js and provider-registry.js. It provides three swappable capability registries: asr, tts, and avatar-media.

The plugin integrates the character manifest and dialogue pipeline, and supports keyless browser TTS and browser speech capabilities. Character preview images and opening videos are shipped in the assets/ directory. Emotion clips come from an external character pack. If missing, the UI automatically falls back to a static avatar.

Installation and Activation

Before installation, make sure DeepSeek Harness is installed and the runtime environment requirements are met. The following are the specific installation and activation steps:

  1. Install the plugin
    Use the plugin management command to add local or path-based live-talk to the avatar configuration file:
    dsh plugin --profile avatar add /path/to/live-talk
  1. Verify the configuration
    Run the command to check whether live-talk was loaded successfully in the configuration file:
    dsh --profile avatar --dump-config
  1. Start the Web UI
    Start the DSH web interface:
    dsh --profile avatar web
  1. Select the conversation view
    In the top view tabs of the conversation in the Web UI, select the Live Talk tab to enter the character conversation interface.

Development and Testing

If you need to perform secondary development or verification testing on this plugin, follow these steps:

  1. Build the code
    Generate the lib/ directory from the src/ directory:
    npm run build
  1. Run unit tests
    Run all unit tests (including the build step):
    npm test
  1. Run integration tests
    Run the full integration tests. Note: This step requires the DSH_REPO environment variable to point to the deepseek-harness source directory, and DSH_HOME must also be set:
    npm run test:integration

Notes

When deploying and using Live Talk, keep the following facts in mind:

  • Environment requirements: Node.js version >= 22.13.0 is required.
  • Verified version: The current plugin has been verified with DSH version 0.1.0-rc.5.
  • Installation methods:
    • When installing directly from GitHub, you need to pin a specific commit and allow the prepare build script.
    • For versions published via npm or tarball, the lib/ directory is prebuilt, so no build permissions are required.
  • Credential management: All third-party Provider credentials are passed through DSH’s ctx.credentials and should not appear in patch files or configuration files.
  • Integration testing: test:integration requires a local DSH source environment.

References