Duplicate File Cleaner
Paste the following prompt into your AI chat to install this skill:
To install this skill in your AI assistant, follow the guide at https://skillhub.cn/install/skillhub.md and use the specification: @user_b3d06a4a/duplicate-file-cleaner.
About this skill
Problem: The Dilemma of Duplicate Files on Disk
A common yet challenging issue in daily development or operations is disk space being cluttered with duplicate files (e.g., temporary build artifacts, redundantly downloaded dependency packages, multiple backups of codebases). Manually finding these files is time-consuming and carries a high risk of accidental deletion, especially when filenames differ. We need a tool that can accurately identify and safely handle these duplicates.
Solution: An Intelligent Scanning and Safe Cleanup Workflow
This skill (@user_b3d06a4a/duplicate-file-cleaner) provides a structured workflow to address this problem by executing a series of scripts, forming a closed loop from identification to cleanup.
1. Multi-Dimensional Duplicate File Identification
The core tool is scripts/duplicate_scanner.py, which supports three scanning strategies:
* comprehensive: A combined strategy using both content and metadata, offering the highest accuracy. Recommended.
* content: Identifies files based solely on content (e.g., hash values), ensuring byte-for-byte consistency.
* metadata: A fast filter based on filename, size, and modification time, which may include false positives.
You can specify the target directory with the --directory parameter and filter specific file types (e.g., jpg,png,pdf) using --extensions.
2. Risk Assessment and Intelligent Decision-Making
After scanning, scripts/risk_assessor.py evaluates the operational risk, outputting a risk level from low to high along with specific contributing factors. This prevents blind deletion. Subsequently, the skill's "agent" interprets the report, recommends strategies based on the risk level, identifies high-value files (e.g., the earliest created version, critical system files), and generates a deletion list annotated with high-risk items.
3. Safe File Organization and Deletion
For non-duplicate files that need categorization, scripts/file_organizer.py can be invoked to organize them by type (type), date (date), or size (size) with a preview. For the final deletion, scripts/safety_manager.py provides four layers of protection: automatic backup, operation confirmation, log recording, and reversal within 30 days, minimizing the risk of data loss.
Boundaries and Important Considerations
- Applicable Scenarios: This skill is best suited for handling duplicate files in user directories (e.g.,
~/Downloads,~/Documents) or project build directories. It is not recommended for critical system directories (e.g.,C:\Windows\System32on Windows or/usron Linux). - Risk Control: When handling high-risk directories or if uncertain, always use the
--dry-runparameter for a preview that does not execute actual deletion. - Strategy Selection: The
metadatastrategy is fast but may include non-duplicate files, so results should be reviewed carefully. For critical cleanup tasks, thecomprehensivestrategy is recommended. - Backup Management: The default backup function occupies space. Please regularly manage old backups via the cleanup mechanisms in
scripts/safety_manager.py.
Use Cases
- After a project concludes, the need arises to clean up multiple duplicate build artifact directories and backup files to free up server disk space, while ensuring no important codebases or configuration files are mistakenly deleted.
- A photographer or designer has accumulated tens of thousands of images and asset files, requiring quick identification and safe cleanup of identical or similar multiple versions through content hashing and metadata analysis to free up storage for new projects.
- A system administrator conducting periodic reviews of `/home` user directories needs to identify duplicate documents and log files generated from downloads and copies, organize them according to strategy (e.g., archive by date), and generate a risk report for team review.
- A developer needs to quickly scan an entire workspace to find duplicate dependency packages (e.g., node_modules, .jar files) and test data, reducing repository size through safe deletion.
Best For
- An operations engineer responsible for managing server storage space, needing to regularly perform duplicate file cleanup and archiving tasks with risk controls.
- A software developer who needs to archive or clean up large project directories after project completion, seeking to safely delete redundant build caches and test data.
- A creative worker managing a personal or work media library (e.g., photos, videos, design assets) who needs to organize and free up storage space.
- An IT manager responsible for maintaining a team file server or cloud storage, needing to develop and implement file cleanup policies.
Related Skills
Comprehensive network engineering skill covering fault troubleshooting, technical consulting, and architecture design across routing and switching, wireless, security, cloud computing, and optical transport.
Provides 9 application modules based on GB/T 33000-2025 and DuPont safety methodologies for diagnosing, standardizing, and building culture in enterprise safety management systems.
A security-first vetting protocol for AI agent skills, used to thoroughly check code and risks before installation.
Automatically scans skill packages before and after installation to block malicious or non-compliant code, acting as a security gatekeeper for skill platforms.