Website Replication Agent

Problem It Solves

When replicating a front-end site or building a static mirror, the hard part is not just “downloading HTML.” Pages may rely on JavaScript rendering, CSS/fonts/images may be cross-origin, links may still point back to the source host, and missing assets can look fine locally but break after deployment. Replication Agent turns this into a reviewable workflow: confirm authorization first, then crawl, render, download assets, rewrite links, and generate quality reports before allowing local preview or static deployment.

How It Works

• Authorization gate: before running a mirror, it requires confirmation that the user owns the target site or has explicit permission. If authorization is unclear, it stops and waits for confirmation. It refuses to copy private areas, bypass paywalls/authentication, clone login/payment flows, or impersonate another brand.
• Crawl, render, and rewrite: scripts/replication_agent.py handles crawl, render, download assets, and rewrite links, then generates reports.
• Quality signals: inspect quality_report.json, crawl_table.json, and resource_table.json to check ready_for_release, page success rate, whether CSS/JS/image/font/video/document assets remain remote or missing, and whether internal links still point to the source host.
• Preview and deployment: scripts/serve_replica.py previews the aggregated localhost output; scripts/deploy_static_mirror.py can upload generated static output and Nginx config over SSH. Deployment happens only when quality checks pass and the user explicitly requests it.
• Configuration and checklist: configs/replication_agent.example.json provides a minimal template; references/usage.md and references/release-checklist.md contain command examples, configuration details, and release checks.

Boundaries

This skill fits authorized site mirroring, internal previews, and static backups. It is not for scraping unauthorized content, bypassing login/paywall flows, copying brand assets, or building impersonation pages. Treat quality_report.json as the release gate: do not recommend public deployment if assets remain remote, console errors block rendering, same-domain 404s appear, or success rate is below threshold. For incremental updates, omit --force-refresh, but the core order remains authorization first, replication second, and acceptance last.