Preface

In DSH’s plugin-based development, a common issue is the lack of a stable entry point between model conversations, web page previews, and media generation.
For example, you might want to generate images, videos, voice, or music in the Web UI, or have the model inspect frontend screenshots, charts, and design drafts, while also making the generated results directly previewable in conversation cards.

AtropinolTT/dsh-guide-dog is a multimodal plugin for the DSH web profile, built on MiniMax capabilities, combining visual inspection, media generation, voice interaction, and same-origin previews into a single plugin.

What This Is

It is maintained by AtropinolTT and licensed under MIT.
The core positioning of this plugin is to add multimodal capabilities to the DSH Web frontend, including visual inspection, media generation, voice interaction, and Web UI previews.

Key capabilities include:

  • MiniMax VLM visual description/inspection: guide_dog_vision, guide_dog_inspect
  • Image, video, voice, music, text, and web search generation
  • Web UI same-origin preview/playback: /guide-dog/media/<file>
  • Settings page includes authentication status and speak tester
  • guide_dog_speak reuses the audio-conversation and speech-mmx skill pipelines, falling back to mmx speech synthesize when missing
  • System prompt automatically invokes the visual inspection tool: guide-dog-vision, order 110
  • Real-time conversation mode: VAD/push-to-talk, streaming sentence-level TTS, barge-in, write-operation consensus protection, progress indicators, Chinese/English interface
  • Accessibility mode is in the planned stage, with the a11y configuration block reserved

Core Features

Visual Description and Inspection

The plugin provides two vision-related tools:

  • guide_dog_vision: General visual description
  • guide_dog_inspect: Structured visual inspection

This is suitable for scenarios where the model needs to “look” at an image before continuing work, such as inspecting charts, screenshots, frontend design drafts, or verifying whether generated images match expectations.

The plugin also mounts a system prompt segment guide-dog-vision with an order of 110, used to automatically invoke the corresponding tools in tasks requiring visual inspection.

Media Generation

The plugin provides multiple generation capabilities:

  • Image generation
  • Video generation
  • Voice generation
  • Music generation
  • Text generation
  • Web search

Generated results enter the plugin’s media directory and can be previewed or played in the Web UI via a unified path.

Web UI Preview

After generating media, the plugin provides a same-origin preview path:

/guide-dog/media/<file>

This path is used to render or play the generated results in conversation cards. The response should include the returned url field so the frontend can display an accessible media address.

Voice and Real-Time Conversation

guide_dog_speak handles voice requests. It reuses the existing audio-conversation and speech-mmx skill pipelines; when the relevant skill scripts are missing, it falls back to mmx speech synthesize.

Real-time conversation mode provides the following capabilities:

  • VAD or push-to-talk
  • Streaming sentence-level TTS
  • Barge-in
  • Write-operation consensus protection
  • Progress indicators
  • Chinese/English interface

The settings page includes authentication status and a speak tester for verifying the voice pipeline.

Accessibility Mode

Accessibility mode is currently in the planned stage.
The repository retains the a11y configuration block for future expansion.

Installation and Enablement

The installation command is:

dsh plugin --profile web add github:AtropinolTT/dsh-guide-dog

This command includes --profile web, indicating installation under the web profile.

For static bundle deployment, the steps are:

  1. Edit plugin-host.js / plugin-client.js
  2. Run deploy/convert_bundle.py
  3. Run deploy/publish.py
  4. Restart DSH

The bundle is resolved when DSH starts, so DSH must be restarted after deployment. The current deployment does not use dynamic plugins, approval cards, or per-session instances.

Typical Usage

Visual Inspection Example

A typical visual inspection workflow is: first generate or create an image file, then invoke the inspection tool, and finally iterate based on the results.

Example: After creating chart.png, invoke:

guide_dog_inspect({ image: "chart.png", focus: "figure" })

Adjust the chart based on the returned inspection results, then re-inspect. Once confirmed to be correct, include the previewable url in the final response.

Previewing After Media Generation

After generating media, preview it in the Web UI via the following path:

/guide-dog/media/<file>

Additionally, include the returned url field in the response so users can access the generated results in conversation cards.

Voice Requests

Voice requests are routed to guide_dog_speak.

If you have already configured the audio-conversation and speech-mmx skill pipelines, guide_dog_speak will reuse these pipelines; if missing, it falls back to mmx speech synthesize.

Applicable Scenarios and Notes

This plugin is suitable for developers who need to add multimodal capabilities to the DSH web profile, especially in the following scenarios:

  • Need the model to inspect images, screenshots, charts, or frontend designs
  • Need to generate images, videos, voice, music, or text in conversations
  • Need to preview generated results directly in the Web UI
  • Need voice input, voice output, or real-time conversation mode

Notes before use:

  • The plugin runs with the current DSH process permissions; review the source code and license before installing.
  • Media is globally stored at ~/.dsh/guide-dog, and since 2026-08-16, it has been a single instance for the entire web profile.
  • The media service uses an extension allowlist, basename-only lookup/traversal guards, and supports byte-range requests.
  • plugin-source.js is a legacy reference file and is not part of the current deployment flow.

Conclusion

The value of AtropinolTT/dsh-guide-dog lies in integrating visual inspection, multimodal generation, voice interaction, and Web UI previews into the DSH web profile, bringing plugin capabilities closer to real-world development workflows.

Repository address:

https://github.com/AtropinolTT/dsh-guide-dog