Preface

DeepSeek Harness (DSH) integrates different model backend capabilities into a unified inference framework through its plugin system. During use, developers often need to connect to multiple Serverless inference services to switch models or extend inference capabilities. The dsh-llm-friendli plug-in is designed to meet this need, adapting FriendliAI’s serverless LLM capabilities into the DSH ecosystem.

What Is This

This is a FriendliAI serverless LLM adapter for DeepSeek Harness. The plugin is maintained by Lee-Si-Yoon. It provides DSH with an OpenAI-compatible chat completion interface, supports dynamic model discovery and reasoning capabilities, and allows developers to invoke FriendliAI services through a standard protocol.

Core Features

  • OpenAI-compatible chat completions: Provides a chat completion interface compliant with the OpenAI protocol, facilitating integration.
  • dynamic model discovery: Supports dynamic discovery of available model resources.
  • reasoning: Supports processing reasoning tasks.

Installation and Enablement

No official installation command is currently provided. It is recommended to obtain installation instructions from the plugin directory or source code repository. When enabling it, ensure that the DSH process has sufficient permissions to call FriendliAI services.

How to Use

The plugin exposes service capabilities through an OpenAI-compatible interface. After pointing DSH configuration to this adapter, you can use the standard chat completion interface for requests and obtain inference results using FriendliAI’s serverless capabilities.

Use Cases and Notes

  • Use cases: DSH users who need to integrate FriendliAI serverless inference capabilities, or agent development scenarios requiring dynamic model discovery and reasoning features.
  • Notes: The plugin runs with the permissions of the current dsh process. Be sure to check the source code and license before installation to confirm that usage requirements are met.

Conclusion

The plugin provides DSH users with a standardized way to connect to FriendliAI. For more details and configuration, refer to the plugin directory or source code repository.