Preface

Running agents in DeepSeek Harness (DSH) distributes model calls across normal conversations, retries, context compression, and different provider routes. While provider consoles show billing, it’s difficult to correlate “which session, which route, how much compression accounted for”; local session projections may also mix usage from older versions that can’t be attributed daily. To perform local budget alerts, trend comparisons, or single-session trajectory audits, one typically has to write custom statistical scripts, which are hard to integrate with the DSH Web settings page.

dsh-token-usage is a community-maintained DSH plugin published by LeemanCheung. It passively observes Host-side events under the Web profile, persistently records four token buckets, and provides a dashboard, budget, public rate estimation, aggregated exports, and on-demand AI usage analysis and session trajectory reports. The plugin currently has 12 stars on GitHub and is categorized as “Memory” on SkillHub.

What Is This

One-line positioning: A local-first token observability, budget, and trajectory audit plugin for DeepSeek Harness.

Its core problem is: Without intercepting model requests, it accounts for four buckets—uncached input, output, cache reads, and cache writes—into a recoverable session projection, and centrally displays trends, budgets, efficiency metrics, and session-level drill-downs on the Settings → Token Usage page. USD figures are estimated using built-in static public rates and do not impersonate provider bills; AI analysis and trajectory reports require explicit user initiation and only send bounded aggregated DTOs or whitelisted trajectory metadata.

The license is MIT. Full documentation is available in the GitHub repository and the SkillHub directory page. SkillHub is an independent community directory with no official affiliation with DeepSeek / High-Flyer.

Core Features

Precise Accounting and Multi-Dimensional Aggregation

Observes normal model requests, retries, and compression events on the Host side, building a recoverable session statistics projection. reasoningTokens are included in the output and not double-counted. Streaming usage is first recorded as a temporary value; within the same attempt, the final message overwrites it. Retries and context compression are counted independently. Aggregation is available by provider/model, session, and UTC date. Old usage that cannot be attributed is disclosed separately without breaking total conservation.

Statistical metrics (from README):

Metric Calculation Method
Input Tokens uncachedInputTokens + cacheReadTokens + cacheWriteTokens
Total Tokens Input Tokens + outputTokens
Cache Reads as % of Input cacheReadTokens / Input Tokens (structural ratio of tokens, not request-level cache hit rate)
Compression Tokens Sum of all four buckets from compaction/summary provider usage
Tokens per Model Attempt (Total Tokens - Compression Tokens) / assistantRequests; retries count as independent attempts

The dashboard provides eight overview metrics: total, input, output, cache structure, public cost, cost avoided via cache reads, rate coverage, and number of sessions with usage. A 30-week UTC daily heatmap supports hovering over the four buckets and drilling down into sessions by day. Periodic trends can be toggled between 7/30/90-day windows to view totals, period-over-period changes, active days, and peak days.

Running rate, budget forecast, and anomaly detection are only enabled when daily buckets are complete and reliable, preventing underestimation from synthetic dates in older versions. The running rate uses the daily average of the last 7 full UTC days (excluding today), multiplied by 30 for a rolling forecast; anomaly detection compares yesterday with the median/MAD of at least 5 active baseline days from the previous 28 days, and anomalous days allow drilling into session contributions.

Budget and Agent Efficiency

A rolling 30-day token budget is written to local DSH settings (token-usage.rolling30DayBudget). When the budget is enabled and daily coverage is complete, the interface shows consumption percentage, a 30-day forecast based on the current running rate, and potential overspend alerts; filling 0 or clearing disables it. The plugin only displays evidence and does not block model calls.

The Agent efficiency section shows the number of model attempts, tokens per attempt, compactions per 100 attempts, compression token percentage, cache reads as % of input, top 1/top 3 route concentration, and unattributed percentage.

Public Price Estimation

Costs are calculated using built-in static public rates (USD per 1M tokens). The current README lists matching ranges including OpenAI’s gpt-5, gpt-5-mini, gpt-5-nano, gpt-4.1 series, and gpt-4o tags. The interface indicates the rate reference date, token/route coverage, and uncovered items; uncovered routes display —.

AI Token Usage Analysis (On-Demand, Opt-In)

Users select a pre-selected default/first route from the dashboard or choose any currently listable connected provider/model, and the model is only called after clicking generate. The report covers total usage, compression, cache, route contributions, reliable daily trends, peaks, volatility, and token optimization suggestions; it is rendered in Markdown and can be exported. The generation process shows preparation/generation/organization stages; before provider usage arrives, estimates are shown, switching to precise values after. The model directory can be manually refreshed; a single provider enumeration failure does not affect other routes. Aggregated AI usage reports are not persisted.

Session Trajectory Analysis

Can be initiated from the session list in the settings page or the action area in the conversation page title, supporting both live and cold sessions. Analysis covers call nodes, retries, compression, tool reliability, rate, lifecycle, and token reconciliation, including compliance control audit (approval request and decision pairing statistics). The report distinguishes between observed evidence, risk assumptions, and unavailable evidence. Trajectory reports are saved locally in the current browser’s localStorage (key: dsh-token-usage.trajectory-history.v1) with a maximum of 24 entries; the interface filters by the current session and allows deletion.

Aggregated Exports

Exports JSON v2, daily CSV, and model CSV without session content and titles. CSV cells are protected against formula injection; JSON and model CSV include public rate coverage and covered route estimates.

Design Boundaries

  • Passive Ledger: Does not intercept requests or rewrite routes.
  • On-Demand AI: Prompts, responses, titles, paths, tool parameters, and raw provider/model do not enter model evidence.
  • Estimation, Not Billing: USD figures and approval statistics do not replace provider bills, policy enforcement, or certification audits.
  • Privacy: Persistent projection only saves statistical data; private RPC only allows loopback pages.

Installation and Enabling

The plugin requires the DSH Web profile with the full Client service mounted, and depends on DSH 0.1.0-rc.6 series session, LLM, settings, projection, and Web UI services. CLI or non-Web profiles do not provide the dashboard.

Official installation command is as follows. After installation, restart the current dsh web process and refresh http://127.0.0.1:3080, then open Settings → Token Usage.

dsh plugin --profile web add github:LeemanCheung/dsh-token-usage

For local source development, execute in the directory above the plugin directory:

dsh plugin --profile web add ./dsh-token-usage

Uninstallation command:

dsh plugin --profile web remove dsh-token-usage

After uninstallation, restart dsh web and refresh the page. Uninstallation removes the plugin mount, not the data reset; to reduce local residuals, first delete trajectory history reports, clear the budget to zero, then handle projection data according to DSH’s own session/cache policies.

Typical Usage

  1. Complete installation and restart dsh web.
  2. Open Settings → Token Usage.
  3. View total, input/output, cache structure, and public USD estimates on the overview cards.
  4. Hover over the 30-week heatmap to see the four buckets, click a cell to drill down into that day’s sessions.
  5. Toggle 7/30/90-day periodic trends to compare period-over-period changes and peak days.

Setting a Budget and Observing Running Rate

Enter a token limit in the 30-day budget area (writes to token-usage.rolling30DayBudget). When all sessions included in statistics have complete real daily buckets, the interface shows rolling consumption percentage, a 30-day forecast based on the current running rate, and overspend alerts. When coverage is incomplete, it indicates which metrics are unavailable.

Generating an AI Usage Optimization Report

  1. In the AI Token Usage Analysis section, confirm or refresh the model directory.
  2. Select a connected provider/model for analysis (default uses the pre-selected route from the directory).
  3. Click generate, wait for the preparation/generation/organization stages to complete.
  4. Read the Markdown report, export if necessary.

The selected route is only called after the user clicks generate; if the directory fails, it can be retried, and it will not silently switch to a default model.

Single-Session Trajectory Analysis

  1. Click trajectory analysis in the first column of the session record table, or access the same process from the action area in the conversation page title.
  2. View four sets of deterministic summaries: call nodes, retries, compression, tools, and token reconciliation, etc.
  3. Export the report or filter/delete old reports in the browser’s local history by session (maximum 24 entries).

Exporting Aggregated Data

Choose JSON v2, daily CSV, or model CSV at the aggregated export entry. Exported content does not include session titles or body text, suitable for external analysis or archiving; note that costs in JSON/model CSV are still public rate estimates.

Use Cases and Considerations

Who Is It For

  • Those who run agents long-term in DSH Web and need local visualization of token structure, route concentration, and compression costs.
  • Those who need rolling budgets and anomaly day alerts but accept the “only show evidence, don’t block calls” design.
  • Those who need to audit trajectories and approval events for single sessions and are willing to trigger AI analysis on-demand while accepting whitelisted metadata boundaries.

Pre-Use Considerations

  • The plugin runs with the permissions of the current dsh process; before installation, review the source code and MIT license to ensure it meets local security and compliance requirements.
  • Only available in Web profile; environments without a Web UI cannot use the dashboard.
  • When older projections lack daily data, historical totals are preserved, but running rate/anomaly metrics exclude incomplete dates.
  • The public rate table has limited coverage; costs for unmatched routes display —; do not treat interface numbers as provider bills.
  • AI usage analysis and trajectory analysis send aggregated or whitelisted data to the selected model; although they do not include session content, they are opt-in operations.

Conclusion

dsh-token-usage collects DSH local token events into a recoverable projection and provides a dashboard, budget, exports, and two types of on-demand analysis reports on the Web settings page. If you are using DSH under a Web profile and need a drill-down usage ledger rather than post-hoc bill checking, install via the commands above and enable in Settings → Token Usage.