Introduction

When developing agents based on DeepSeek Harness (DSH), handling CAPTCHAs on target websites is a common requirement. Manually parsing images or calling third-party services increases development complexity. The xby-captcha plugin wraps external model APIs and provides automatic recognition solutions for common CAPTCHA types.

What It Is

xby-captcha is a DSH plugin categorized under model inference. It provides a toolkit for recognizing CAPTCHAs such as text, slider, rotation, and icon selection, primarily relying on external API services to perform image analysis.

Core Features

The plugin provides the following main capabilities, supporting image URLs, file paths, or Base64 encoded input.

Text Recognition

  • find_chars: recognizes text (numbers, letters, Chinese characters) in an image, returning the text content and confidence.
  • find_chars_for_data_base64 / find_chars_for_data_file: text recognition with Base64 or file path input.

Slider Verification

  • slide_match_target_background: finds the best matching position of the slider image in the background image (with a gap).
  • slide_match_only_background: directly finds the gap position in the background image (with a gap).
  • slide_compare: finds the difference area between the target image and the background image, locating the shadow slider position.
  • slide_match_target_background_for_data_base64 / slide_match_target_background_for_data_file / slide_match_only_background_for_data_base64 / slide_match_only_background_for_data_file / slide_compare_for_data_base64 / slide_compare_for_data_file: Base64 and file path input versions of the above features.

Rotation Recognition

  • rotate_single: recognizes the rotation angle of an image (0-359 degrees).
  • rotate_double: calculates the rotation angle of the target image (inner ring) relative to the background image (outer ring).

Icon and Text Detection

  • detect_icon: based on the icon order in the prompt image, detects the corresponding icons in the target image and returns coordinates in order.
  • detect_text: based on the text order in the prompt image, detects the corresponding text in the target image and returns coordinates in order.
  • detect_icon_for_data_base64 / detect_icon_for_data_file / detect_text_for_data_base64 / detect_text_for_data_file: Base64 and file path input versions of the corresponding features.

Image Comparison and Math Calculation

  • compare_image: calculates the similarity between two images (0-1 score).
  • compare_image_for_data_base64 / compare_image_for_data_file: Base64 and file path input versions of the corresponding features.
  • solve_math: recognizes math expressions (addition, subtraction, multiplication, division) in an image, automatically calculates the result, and returns the expression and result.
  • solve_math_for_data_base64 / solve_math_for_data_file: Base64 and file path input versions of the corresponding features.

Installation and Enabling

Using the official installation command adds the plugin to a specified Profile:

dsh plugin --profile web add github:xby_skill/xby-captcha

Typical Usage

  1. CAPTCHA Recognition: input a CAPTCHA image URL or file path, then call find_chars to get the recognition result.
  2. Slider Verification: input the slider image and background image, then call slide_match_target_background to get the gap coordinates.
  3. Rotation Calculation: input the target image, then call rotate_single to get the rotation angle.
  4. Icon Sorting: input the prompt image and target image, then call detect_icon to get the ordered coordinate points.

Use Cases and Notes

This plugin relies on external API services. When using it, always comply with the terms of use and applicable laws and regulations of the target website or system, and ensure use only under permitted conditions.
Before installation, it is recommended to check the source code and license (MIT).

Conclusion

xby-captcha lowers the barrier for agents to handle CAPTCHAs by providing standardized interfaces. For more details, visit the Plugin Directory or the GitHub Repository.