Introduction

Common approaches to showing an AI a screenshot fall into two categories: either manually converting the image content into text descriptions, or uploading the image to a cloud multimodal interface. The former is time-consuming and prone to losing details such as position and layout; the latter requires considerations regarding privacy and cost.

DSH’s philosophy is “everything is a plugin,” and visual capabilities can be seamlessly integrated as such. What @zenk/vision introduced in this article accomplishes exactly that: simply paste a screenshot directly into the chat dialog, and the AI can view and analyze it without the image ever leaving your machine.

What is it

@zenk/vision is a local vision plugin for DSH, maintained by kaaaaahn, released under the MIT license, and currently at version 0.3.1. It integrates two local vision pipelines into DSH: macOS Vision handles OCR, while qwen3-vl on Ollama handles semantic understanding.

Core Features

  • OCR with Pixel Coordinates: Based on macOS Vision, it identifies text while providing pixel positions, allowing for direct UI positioning by coordinates.
  • Semantic Understanding: Running qwen3-vl via Ollama enables comprehension of screen layout, elements, and anomaly areas.
  • Zero Configuration: Automatically detects the environment and selects a model based on memory (8G → 2b / 16~32G → 4b / 32G+ → 8b). Automatically installs Ollama if it is missing.
  • Progressive Availability: Models are downloaded in the background (1.8~5.7GB), and OCR is available during the download process.
  • Fully Local: Free and offline; images never leave your Mac.
  • Image Upload Bridge: Converts image blocks to text, allowing usage in text-only channels.

Installation and Usage

Use the installation command provided in the README:

dsh plugin --profile web add "github:kaaaaahn/dsh-vision#v0.3.1"

This command pulls version v0.3.1 from GitHub. After installation, restart DSH to apply changes; for the desktop version, simply replace --profile web with --profile desktop.

Please note that the plugin marketplace entry (source URL: https://kaaaaahn.github.io/dsh-vision/catalog/source.json) relies on an npm release and is currently not live. For now, please use the GitHub command above for installation.

Upon first use, the plugin will automatically download the models in the background (1.8~5.7GB), and OCR is available during the download. The environment depends on macOS Vision and Ollama; Ollama is automatically installed if missing. For environment details, refer to docs/ollama-setup.md in the repository.

Typical Usage

The most direct usage: paste an image in the dialog and send it; the AI will automatically analyze and reply. Screenshot, paste, send—no extra commands required.

The second method is to have the AI call a tool to analyze local files:

vision_analyze(file_path=..., describe=true)

file_path specifies the local image path, and describe=true enables semantic description. This is suitable for analyzing images already on the disk, such as asking the AI to describe the layout and elements of a screenshot.

Use Cases and Notes

Suitable Scenarios:

  1. Paste screenshots directly during daily conversations and let the AI answer based on the image, saving manual description.
  2. When working on UI-related tasks, use OCR with pixel coordinates to locate elements.
  3. Environments where privacy is sensitive and screenshots are not allowed to leave the machine.

Pre-Use Notes:

  1. The plugin relies on macOS Vision and is designed for Mac environments.
  2. The first use will download models totaling 1.8~5.7GB; pay attention to disk space and network; refer to docs/ollama-setup.md for model selection.
  3. Regarding security, the plugin runs with the permissions of the current DSH process. It is recommended to review the source code and license before installation. This project is under the MIT license, with core code concentrated in lib/index.js (plugin entry point, including vision_analyze, vision_setup, image bridge, etc.) and lib/vision_analyze.swift (OCR and Ollama semantic description scripts distributed with the package), which can be reviewed directly.
  4. If you encounter issues, check docs/troubleshooting.md first, which contains 22 FAQs.

Conclusion

What @zenk/vision does is straightforward: it integrates macOS’s built-in OCR and local multimodal models into DSH, making “showing an AI a picture” as simple as a paste, while ensuring the image never leaves your machine. For DSH users who frequently need to read screenshots, this is a practical addition.

  • Plugin Directory Page: https://www.skillhub.cn/plugins/kaaaaahn/dsh-vision
  • GitHub Repository: https://github.com/kaaaaahn/dsh-vision

Note that the directory page is a community-maintained independent site and has no official affiliation with DeepSeek or Synthflow.