Introduction

The runtime of DeepSeek Harness (DSH) determines whether image input is allowed by checking the inputModalities field in model metadata. When configuring third-party models or custom providers, DSH may be unable to automatically infer whether a model supports images, causing images in messages, read_image tool calls, or attachment uploads to be blocked.

The dsh-model-modality plugin provides an explicit declaration method, allowing users to write modality declarations into the provider settings that own the model, thereby bypassing the runtime’s automatic inference restrictions.

Plugin Scope

This plugin belongs to the DeepSeek Harness ecosystem and is maintained by the user ct-jyjntc. It is primarily used to resolve the unknown input modality issue for third-party models (relay stations / custom providers) by writing inputModalities into the metadata, ensuring that image-related features work properly.

Core Features

  1. Modality declaration: Allows users to declare whether a model supports image input.
  2. Metadata writing: Writes declared modality information into model metadata.
    • llm-deepseek (official / custom baseURL): writes to models[].inputModalities
    • llm-pi-ai (third-party provider directory): writes to models[].input or modelOverrides[].input
  3. Tool support: Provides two main tools for agents to call:
    • declare_model_multimodal(provider, model, supportsImage)
    • list_model_multimodal(provider?, model?)
  4. Optional UI: Provides a browser-side multimodal toggle interface (requires dynamic mounting).

Installation

Install using the official plugin management command:

dsh plugin add ct-jyjntc/dsh-model-modality

Typical Usage

After installation, tools can be called directly in a conversation to perform declaration or query.

Declare model image support:

declare_model_multimodal(provider="raincode", model="kimi-k3", supportsImage=true)
→ { ok: true, verified: true, inputModalities: ["text","image"], ... }

List model modalities:

list_model_multimodal(provider="raincode")

Notes

  • Dependency version: This plugin depends on @deepseek-ai/dsh-tools, and the version range must satisfy >=0.0.1-rc.1 <0.1.0 || >=0.1.0-rc.1 <0.2.0-0. Please check the local environment version before installation.
  • Persistence location: The declared data is written into the provider’s model settings and is independent of the plugin itself. You can view and edit this data in the DSH Models settings page.
  • Browser-side UI: The “Multimodal (Image Input)” toggle in the settings page and Run card panel is an optional feature. Because the client module is not published with this package, if you need to use this UI in the browser, you must have the agent dynamically mount the plugin during the session.

Summary

This plugin resolves the multimodal input recognition problem for third-party models in the DSH environment through explicit declarations. For more details and source code, visit GitHub or see the Community Directory.