About this skill

Problem

When a batch of .docx / .doc files needs cross-checking, manual review makes it hard to see which paragraphs are highly similar, whether repetition comes from different authors, editors, or one reused draft. In bidding, technical proposals, and collaborative documents, reviewers often need duplicate rate, metadata, and paragraph-level evidence at once.

How it works

The skill runs a multi-document duplicate-detection pipeline:
- scan_files.py scans Word files in a directory;
- batch_extract.py extracts paragraph text and document metadata, such as author, last_modified_by, and created/modified timestamps;
- check_duplicates.py calculates paragraph similarity using TF-IDF and cosine similarity, with a default threshold of 0.8 and a minimum paragraph length of 20 characters;
- generate_report.py produces an interactive HTML report with a summary page, heatmap matrix, file comparison, highlighted original-text popups, and CSV export.
The default summary page shows key findings, document attributes, top duplicate file pairs, author/editor analysis, and cross-author duplicate statistics. After modifying the JavaScript in generate_report.py, report generation runs validate_js.py to check variable declarations and brace pairing in buildSummary().

Boundaries

This skill is focused on paragraph-level duplicate detection across Word documents, not full plagiarism certification. .doc metadata extraction is best-effort. Threshold and minimum-length settings affect recall; short paragraphs or heavily rewritten text may be missed. It is useful for first-pass checks of bid documents, technical proposals, and contract attachments, but should not be treated as the sole copyright or originality decision.

Screenshots

Use Cases

Bid teams compare multiple technical proposals before submission to locate repeated paragraphs and author or editor signals.
Legal assistants review contract attachments, check cross-document reused clauses at a threshold, and export CSV for audit.
Project managers review vendor proposals and use the heatmap to identify highly similar sections across files.
Document owners investigate template reuse by extracting author, last editor, and timestamps to infer the base draft.

Best For

Bid document leads who need a quick pre-submission check for large reused sections across proposals.
Legal or compliance assistants who need to aggregate repeated clauses by file and author for review.
Technical project managers who need to compare similar sections and their likely source across vendor or internal proposals.
Document engineering staff who modify report JavaScript and need variable-reference validation before publishing the HTML report.