Omni Reader Multimodal Document Parsing

Problem

Multimodal inputs are often scattered across PDFs, Office files, images, web pages, audio, and video. When engineers need searchable text for agents, common issues include lost reading order, broken table structure, unreadable scans, and missing visual semantics in audio or video.

How It Works

omni-reader parses inputs into markdown or clean text. PDFs and Office documents preserve reading order and table structure; images and scans use OCR plus visual understanding; audio is transcribed with ASR; video combines ASR, keyframe visual analysis, and multimodal fusion. URLs can be parsed directly, while local files are processed through user-specified paths. Status, result reading, cancellation, and export actions are exposed through MCP tools, and backend services can also integrate via HTTP API.

Boundaries

Each run processes one file, with a single-file limit of roughly 256 MiB. Large files should be split, and video should be chunked into 15–30 minute segments. The default is no_store=true, so source files and results are not uploaded to the service; however, actual parsing depends on the remote service and may be affected by network, quota, peak hours, or service status. Unreachable addresses such as oss:// URLs should first be converted to signed HTTPS URLs.