Introduction

In the development of DeepSeek Harness (DSH), when input content (such as logs, API responses, or data reports) is too large, it can fill the context window before the model processes the instructions. The existing dsh-compaction-tool-result-pruner is a post-processing pruning tool, while dsh-token-diet is a pure-function toolkit for structure-preserving summarization before content enters the context. It reduces token consumption at the source with zero dependencies.

Core Features

The plugin provides five main tools. Choose based on the input type:

  1. diet_text: For logs, documents, and crawled page content. Extracts head + tail + line count + high-frequency keywords + token estimation.
  2. diet_json: For API responses and configuration files. Extracts the key skeleton + types/key counts/array lengths + sampled values.
  3. diet_csv: For data files and report exports. Computes column type/distinct/min/max/avg statistics + sampled rows.
  4. diet_estimate: For arbitrary text. Estimates tokens + provides context entry recommendations (decide to read the full text, slim it down, or skip it).
  5. diet_stats: Cumulative savings statistics. Aggregates savedTotal across calls.

Savings Feedback

Each call to a slimming tool (diet_text/json/csv) returns a saved field with actual savings data:

{
  "kind": "csv",
  "rowCount": 8000,
  "colStats": [...],
  "saved": {
    "originalTokens": 38563,
    "outputTokens": 196,
    "savedTokens": 38367,
    "savedPercent": 99.5
  }
}

Call diet_stats to view cumulative totals:

{
  "calls": 3,
  "originalTotal": 110272,
  "outputTotal": 1102,
  "savedTotal": 109170,
  "savedPercent": 99
}

Compression Configuration

The plugin registers the token-diet settings namespace. It supports three presets and a manual compression ratio.

Three Presets

Set preset in the Web UI or settings file:

  • light: Retains more details; suitable for code review. headChars: 2000, maxKeys: 25.
  • balanced (default): Balances skeleton completeness and token reduction. headChars: 800, maxKeys: 12.
  • aggressive: Maximally token-efficient; suitable for long conversations. headChars: 300, maxKeys: 6.

Manual Compression Ratio

Set compression: 0-99 to directly specify compression strength:

  • 0: No compression (mode: "raw"), useful as a comparison baseline.
  • 99: Maximum compression.
  • Within this range, the system linearly interpolates over parameters such as headChars and maxKeys.

Priority chain: explicit tool parameters > tool compression > settings compression > settings per-item overrides > preset defaults.

Safety Model

  • Pure functions: No file reads, no network access, no eval execution.
  • Input limit: 512KB; oversized input returns an error directly instead of being truncated.
  • Output format: Compact structured JSON string.
  • Unicode safety: Truncates by code point without splitting surrogate pairs; supports emoji and rare characters.

Installation and Enablement

Include it in cordis.yml or dsh.profile:

plugins:
  - id: tool-token-diet
    name: 'dsh-token-diet'

For a lightweight version (single tool, no settings dependencies), use dsh-token-diet/lite:

plugins:
  - id: tool-token-diet-lite
    name: 'dsh-token-diet/lite'

Technical Requirements

  • Node version: ^22.19.0 || >=24.0.0.
  • Dependencies: @deepseek-ai/cordis, @deepseek-ai/dsh-tools.

Summary

dsh-token-diet replaces full-text input with structure-preserving summaries, actively slimming large text/JSON/CSV before it enters the context. With actual token savings feedback and configurable compression strategies, it is suitable for DSH scenarios that process large structured data or long logs.