Introduction

High-throughput agent workflows, parallel sub-agent clusters, and multi-turn tool calls inevitably hit upstream API rate limits (HTTP 429, wasted RPM/TPM, daily quotas, or burst interruptions). In a standard DeepSeek Harness deployment, exhaustion of a single API Key can break the entire agent execution chain, requiring manual intervention and corrupting session replay state.

dsh-key-rotation is an enterprise-grade, transparent API Key pooling, preemptive rate-limiting, and cross-provider failover engine natively built on the Cordis microkernel architecture.

Feature Overview

The plugin maintains a separate Key pool for each provider and automatically switches to the next Key on quota/rate-limit errors.

  • API Key Rotation: Manages key pools per provider and automatically creates cloned routes.
  • Preemptive Rate Limiting: Skips already throttled Keys before sending network requests.
  • Cross-Provider Failover: Smoothly escalates to a backup provider when the entire Key pool is exhausted.
  • Circuit Breaking: Opens the circuit after consecutive failures; requests fail fast until cooldown.
  • Webhook Notifications: Supports aggregated alerts for easier monitoring.
  • Self-Healing: Periodically probes in the background and automatically recovers failed Keys.
  • Latency-Aware Routing: Supports round-robin, least-loaded (concurrency), and lowest-latency (P95 latency) policies.
  • Native Bilingual Support: Built-in English and Chinese UI dictionaries.

Core Mechanisms

Unlike simple routing proxies that modify provider identifiers, the plugin hooks into ctx.credentials.resolve at runtime and intercepts llm/stream.

  • Session Consistency: Provider identity remains unchanged, ensuring 100% consistency for agent replay state, multi-turn calls, and tool mode descriptions.
  • Concurrency Control: Balances active streams across Keys via the least-loaded policy to prevent burst saturation. Deterministic try ... finally blocks ensure zero concurrency leaks.
  • Error Classification and Handling: Explicitly classifies and handles 408, 425, 429, 5xx, Socket, and gRPC errors.
  • Soft and Hard Backoff: Distinguishes between transient infrastructure failures (soft backoff) and hard quota errors (exponential backoff).
  • Rate Limit Calculation: Uses a Token Bucket algorithm with O(1) time complexity and zero-allocation memory.

Resource and State Management

  • Memory Management: Applies 30-day rolling-window data pruning, automatically cleans up stale Keys, and limits memory growth.
  • Clock Safety: Cooldown and circuit-breaker durations use the process monotonic clock, preventing NTP adjustments from causing remaining time to move backward.
  • State Safety: Atomic I/O helpers ensure crash-safe writes; corrupted JSON does not overwrite prior in-memory state.
  • Zero Blocking: Webhook notifications are handled with a bounded queue and backoff; stream rotation never blocks on Webhook HTTP.

Summary

This plugin resolves single points of failure caused by API Key limitations in agent development, providing enterprise-grade stability and a visual management interface.

  • Catalog URL: https://www.skillhub.cn/plugins/GooDAnDReaDY/dsh-key-rotation
  • GitHub: https://github.com/GooDAnDReaDY/dsh-key-rotation
  • License: MIT
  • Node Version Requirement: >=18