Image Scraper

Problem Background

In content creation, acquiring high-quality images is crucial for enhancing work quality. However, manually searching, screening, and downloading images from multiple websites is time-consuming and often inconsistent in resolution and style. Particularly when needing numerous portrait, landscape, or square images, sources are scattered across platforms, and some have anti-scraping mechanisms, making downloads difficult.

How the Skill Works

The Image Scraper skill automates this process by scraping images from multiple high-quality sources. Its core capabilities include:
- Multi-source Support: Primarily uses Baidu Image Search, with Wallhaven and Unsplash as backups, switching automatically based on priority.
- Intelligent Scraping: Uses the baidu_image_scraper_v8.py script to directly extract original image URLs (objurl) from Baidu's search results' data-show-ext attributes, eliminating the need for browser interactions and enhancing stability.
- Size Presets: Offers multiple size presets from 1080p to 8K, easily toggled via command-line parameters like --ratio portrait and --z 7 to meet different platform requirements.
- Deduplication and Cleaning: Built-in MD5 hash deduplication prevents duplicate downloads; supports automatic cleanup of low-quality images and renaming.

Key steps involve running the script with specified keywords and counts, where the script automatically opens browser pages, scrolls to load content, extracts links, and downloads in batches. For Baidu sources, extracting objurl effectively fetches high-definition originals from CDNs like Meituan and iQiyi.

Applicability and Caveats

While the skill offers convenience, users should note:
- Baidu Image Limitations: Requires the agent-browser command, and Baidu links may expire, necessitating batch extraction and immediate downloading.
- Source Quality Variations: Image resolutions differ across CDNs; for example, Baidu images can reach 4K+ for portraits, but sources like Douyin require access via Baidu's CDN cache.
- Download Efficiency: For large-scale downloads (over 200 images), use the --keywords multi-keyword mode and adjust the --scroll parameter to fetch more images.
- Prohibited Platforms: Avoid low-quality or restricted sources like Shetuwang and Zhihu to ensure images match real photography styles.

With proper configuration, this skill effectively addresses image acquisition needs in content creation, but parameters should be adjusted based on actual scenarios to handle scraping challenges.