Introduction¶

In DeepSeek Harness (DSH), if you want an agent to read screen content or describe images, you can attach visual capabilities as tools to an existing session. The dsh-tool-eyes introduced below is a local visual “eyes” plugin of this type.

It provides two tools: screen for visual description and ocr for text extraction.

What is this¶

dsh-tool-eyes is a DSH plugin maintained by go-farther-and-farther with an MIT license.

The core problem it solves is: in a Windows environment, enabling a DSH agent to call a local visual endpoint to describe the screen or an image, or to invoke the Windows built-in OCR engine to extract text character by character.

Core Features¶

screen¶

screen captures the screen or describes an existing image file and returns a textual description via a local OpenAI-compatible visual endpoint.

It accepts an optional image path; if not provided, it captures the screen.

The prompt for screen requires the vision model to only describe visible content, reducing hallucinations for small model names.

Regarding local-first, screen can point to a local endpoint, ensuring images do not leave the local machine.

ocr¶

ocr uses the Windows built-in OCR engine to extract text character by character, zero model, zero GPU, zero cloud.

It also accepts an optional image path; if not provided, it captures the screen.

ocr is completely local with no network required.

Installation and Activation¶

First, confirm the environment requirements:

• Windows 10/11
• Node.js >= 22.19
• DeepSeek Harness >= 0.1.0-rc.6
• screen requires an OpenAI-compatible VLM endpoint, such as llama.cpp, LM Studio, Ollama, or any gateway

This package is only published on GitHub, not on npm. The installation command is as follows:

dsh plugin --profile web add https://github.com/go-farther-and-farther/dsh-tool-eyes

After installation, restart dsh web. The screen and ocr tools will automatically appear in the agent toolkit.

Configuration¶

If you need to specify the visual endpoint used by screen, you can configure baseUrl, model, and timeoutMs under id tool-eyes in the profile’s cordis.patch.yml.

Structure example:

- id: tool-eyes
  name: 'dsh-tool-eyes'
  config:
    baseUrl: <baseUrl>
    model: <model>
    timeoutMs: <timeoutMs>

Here, baseUrl points to an OpenAI-compatible visual endpoint, model is used to specify the model, and timeoutMs is used to specify the timeout parameter.

Typical Usage¶

In a conversation, you can use it like this:

• Ask screen to answer what is on my screen? or describe this image file, optionally adding a prompt to focus on a specific area or detail.
• Ask ocr to execute read all the text on screen or transcribe this error dialog.

Both can pass an image path; if not passed, a screenshot is taken.

The plugin bundles lib/capture.ps1 and lib/ocr.ps1, which can also be run independently:

powershell -NoProfile -ExecutionPolicy Bypass -File lib\capture.ps1 -Prompt "<focus>" -BaseUrl <baseUrl>
powershell -NoProfile -ExecutionPolicy Bypass -File lib\ocr.ps1 -Image <image path>

capture.ps1 accepts -Prompt and -BaseUrl parameters, and ocr.ps1 accepts -Image parameters.

Applicable Scenarios and Notes¶

Suitable for the following situations:

• Using DSH on Windows 10/11 and wanting an agent to read screen text or describe images.
• Wanting ocr to be completely local without a network.
• Wanting screen to point to a local endpoint so images do not leave the local machine.

Notes:

• screen will send the captured image to the configured baseUrl.
• The plugin runs with the current dsh process permissions. You should check the source code and license before installing; the current license is MIT.

Conclusion¶

Following the steps above, dsh-tool-eyes can add two local visual tools to the DSH agent: screen handles description, and ocr handles character-by-character text extraction.

GitHub: https://github.com/go-farther-and-farther/dsh-tool-eyes
Directory page: currently no confirmed link provided.