Introduction

The philosophy of DeepSeek Harness (DSH) is “everything is a plugin.” When developing agents, standard DeepSeek adapters usually handle only text and cannot directly process image input or output. To add “image understanding” or “image generation” capabilities to a DSH agent, you typically need to write a custom backend service. The wumu1111111/dsh-agnes-omni plugin solves this problem by integrating Agnes’ multimodal models as tools into the DSH environment, enabling agents to understand and generate images and establishing a pathway for sending images directly in conversation.

What Is This

wumu1111111/dsh-agnes-omni is a plugin designed for DeepSeek Harness (DSH). It connects to Agnes AI’s free multimodal models through DSH’s Bundle mechanism (including the agnes-2.5-flash vision model and the agnes-image-2.1-flash image model). The plugin does not contain any API keys; all authentication is dynamically resolved through DSH’s credential service, in line with DSH’s modular security design.

Core Features

The plugin mainly provides three core components:

  1. agnes_vision (tool): Provides image understanding capabilities. It supports viewing images, OCR recognition, scene recognition, and chart analysis. This tool sends images to the Agnes model and returns a text description.
  2. agnes_image (tool): Provides text-to-image and image-to-image (image editing) capabilities. Users can generate images using prompts or upload reference images for modification.
  3. Visual bridge (bridge): This is a technical component used to enable image flow in conversations. It allows the agent to receive image blocks in the conversation, decode and save them to a local path before sending them to the text adapter, and replace them with text markers thereby triggering agnes_vision for analysis.

Installation and Enablement

This plugin is a standard DSH Bundle and takes effect once installed into the specified profile. Run the following command in the terminal:

dsh plugin --profile <profile> add github:WUMU1111111/dsh-agnes-omni

After installation, the plugin automatically registers the tools and visual bridge mentioned above through DSH’s Cordis Patch mechanism.

Configuring the API Key

The plugin itself does not contain any API keys. It reads AGNES_API_KEY through DSH’s credential service. You need to configure this key in the environment where DSH runs, using one of the following methods:

  1. Environment variable:
    export AGNES_API_KEY=sk-xxxxxxxxxxxxxxxxxxxx
  1. Configuration file:
    Edit the $DSH_HOME/.credentials.yaml file and add:
    AGNES_API_KEY: sk-xxxxxxxxxxxxxxxxxxxx

Note: The key should not be committed to a code repository.

Typical Usage

After installation and configuration, the agent can use the following capabilities:

  1. Image understanding: Paste or drag and drop images to send them directly in the conversation. The visual bridge automatically handles the images, and the agent then calls agnes_vision to analyze them and provide a text response. You can also provide a local path or URL for an image directly.
  2. Image generation: Give the agent an instruction, for example, “Generate an image of …”. The agent calls agnes_image to generate the image and downloads the result to the current session’s workspace.
  3. Image-to-image: Provide the path to a reference image and specify what changes are needed. The agent generates a new image based on the reference image and the instruction.
  4. Prompt optimization: It is recommended to use English prompts and describe them using the structure [subject]+[scene]+[style]+[lighting]+[composition]+[quality] for more stable generation results.

Optional Configuration

The visual bridge is responsible for intercepting image messages. It requires a target Provider to be specified in order to declare the image input modality. By default, it points to deepseek-official.

If you need to change the routing, you can specify it in the configuration:

- id: agnes-vision-bridge
  name: dsh-agnes-omni/bridge
  config:
    targetProvider: deepseek-official

Notes

  • Security: Neither the plugin source code nor the repository contains any API keys. All request authentication headers are obtained from runtime credential resolution.
  • Image storage: Images are persisted under the .agnes/ folder in the session workspace root directory, keyed by content hash (SHA-256). The same image is written only once.
  • Node fallback: If the main site is unreachable, the plugin automatically falls back to the domestic node api.agnes-ai.cn.

Conclusion

The wumu1111111/dsh-agnes-omni plugin provides a low-cost multimodal integration solution for DSH agent development. With it, developers can process images directly in the conversation flow without maintaining a separate vision backend.