Introduction

AI agent developers often need to refer to external benchmarks to evaluate model performance. Manually visiting livebench.ai to view leaderboards and export data can be cumbersome. This plugin integrates LiveBench’s official leaderboard data into DeepSeek Harness, providing filtering, comparison, and export features, making it easy to view model capability rankings directly in local workflows.

What This Is

A network-enabled utility plugin maintained by addie-ace. It pulls the latest leaderboard version, CSV, and category JSON data from livebench.ai, displays model scores in the DSH interface, and supports filtering and sorting by dimensions such as organization, open weights, and reasoning models.

Core Features

  • Automatic retrieval of the latest leaderboard: Identify the latest leaderboard version published by the frontend, and read the official CSV and category JSON.
  • Scheduled and manual refresh: Checks by default every 60 seconds, with manual refresh supported; on network failure, the cache is retained without fabricating or filling in scores.
  • Multi-dimensional filtering and comparison: Supports filtering by organization, open weights, and reasoning models, along with model search, sub-item sorting, all variants, and fine-tuned models.
  • Comparison and export: By default, only the variant with the highest overall score is retained for each model family, allowing up to 4 models to be compared; task-level scores can be viewed and exported as CSV.

Installation and Enablement

  • Environment requirements: Node.js >= 20.18.1; Windows uses the system curl.exe to access HTTPS and validate certificates.
  • Installation steps: Run the installation script in the source workspace.
powershell -ExecutionPolicy Bypass -File .\scripts\install.ps1 -HarnessHome 'G:\deepseek harness'
  • Installation details: The script only adds files to plugins/dsh-livebench-rankings. Before registration, it backs up the profile’s package.json, lockfile, and patch files. An existing installation with the same name will abort and not be overwritten.
  • Enablement: After installation, refresh the Harness page; if the client entry has not loaded, properly exit and reopen Harness.

Typical Usage

  • Open the interface: Open it through the “AI Rankings” entry at the bottom-left of the Harness input box.
  • Configure refresh interval: Use the Harness plugin configuration editor to set refreshSeconds (range 30-3600, default 60).
  • Access preview: The local preview URL is http://127.0.0.1:4317/dsh-livebench-rankings/.

Use Cases and Notes

  • Use cases: Suitable for developers who frequently need to consult external benchmark data, compare models, and export CSV files.
  • Security and isolation:
    • Unique package name, Cordis row ID, client module, and slot ID: dsh-livebench-rankings.
    • Unique route prefix: /dsh-livebench-rankings.
    • Does not modify global fetch, TLS, proxy, routing, or localStorage; does not read credentials, account data, or chat history.
    • Only downloads pages and data from livebench.ai, and does not execute remote JavaScript; uses Acorn to read static literals and Papa Parse to parse CSV.
    • Cache is stored only in the plugin’s own .cache/rankings.json; concurrent requests within the same instance are merged, and manual refreshes are spaced at least 5 seconds apart.
  • Uninstallation: Use the command dsh plugin --profile web remove dsh-livebench-rankings to unregister the route, clean up periodic tasks, terminate in-flight requests, and remove the dialog and event listeners.

Short Conclusion

This plugin provides native LiveBench leaderboard viewing capabilities for DeepSeek Harness, with data sourced from the official provider and good security isolation. Through the CSV export feature, it can be easily integrated into subsequent automated evaluation workflows.

  • GitHub: https://github.com/addie-ace/dsh-livebench-rankings
  • Directory page: https://www.skillhub.cn/plugins/addie-ace/dsh-livebench-rankings