In DSH projection development, Cold Fold handling for large sessions is a performance bottleneck. In DSH 0.1.0-rc.6, SessionProjectionRegistry.cellFor() performs apply() event by event over the entire session.events when a cell is cold. In testing, Cold Fold for a session with 740,000 events can block for more than 20 minutes, freezing the Node event loop and causing all unary RPCs to time out (30s); the frontend manifests this as signal timed out.

The DSH projection warmup plugin eliminates the above event-loop blocking through live session chunked warmup and fork cache backfill mechanisms.

Core Features

The plugin provides the following capabilities:

  1. Chunked Warmup: Replays cells in event chunks for event counts exceeding the threshold. When a session is entered (created/resume) and its event count exceeds the threshold, it runs ahead of the first synchronous cold fold, yielding the event loop via setImmediate every chunkSize events; after folding completes, it writes directly to registration.cells (WeakMap).
  2. Fork Child Session Cache Backfill: Establishes projection cache rows. For fork child sessions (where header.parentSession exists), it immediately calls cache.write(child) after warmup completes; otherwise, if the session is abandoned, it will never have a cache row, and the next time a historical coldSnapshot is opened, it will fall back to a full read via readFrom(0).
  3. Disk Cold Session Background Row Backfill: Scans large logs missing cache rows, performs streaming parse + fold, and writes rows via putSoft.
  4. API Support: Provides stats and configuration APIs.

Installation

Use the official command to add the plugin:

dsh plugin --profile web add github:orangeofcarl0-sys/dsh-projection-warmup

Typical Usage

The plugin provides HTTP APIs for querying state and configuration:

# 统计与重置
POST http://127.0.0.1:3080/dsh-projection-warmup/api/stats/get
POST http://127.0.0.1:3080/dsh-projection-warmup/api/stats/reset

# 配置获取与设置
POST http://127.0.0.1:3080/dsh-projection-warmup/api/config/get
POST http://127.0.0.1:3080/dsh-projection-warmup/api/config/set

When enabling disk cold-session background row backfill, call the configuration API:

{
  "key": "diskBackfill.enabled",
  "value": true
}

Considerations and Caveats

The plugin runs under the permissions of the current DSH process. Inspect the source code and license before installation.

Known limitations:

  • Synchronous decoding limitation: The synchronous decoding (zstd) inside a full readFrom(0) for cold sessions cannot be safely chunked at the plugin layer. For large files, it can still freeze the event loop for seconds to tens of seconds.
  • Memory risk: Loading extra-large sessions (700,000+ events) already carries a risk of memory OOM. Startup-time backfill can compound with DSH’s own startup projection folding and raise memory peaks.
  • Compatibility: Live session chunked warmup and fork child session cache backfill are enabled by default. Disk cold-session background row backfill is disabled by default and should be enabled as needed.

Summary

This plugin reduces cold-fold time for large sessions from minutes to milliseconds. Through chunking and backfill mechanisms, it significantly reduces the probability of timeouts during history loading and UI interactions. More details are available in the GitHub repository.