Darwin: Autonomous Experiments

Challenges in Skill Optimization

When developing AI Agent skills, developers often face a core dilemma: how to balance the structural规范 of a SKILL.md file with its actual runtime performance? Pure structural reviews (e.g., checking frontmatter or step numbering) cannot guarantee the skill performs well in real scenarios—a perfectly formatted skill might produce low-quality output. Manual iterative optimization is time-consuming, prone to local optima, and lacks automated evaluation and safe rollback mechanisms.

Autonomous Experiment Loop: Core Mechanism

The Darwin skill addresses these issues through an autonomous experiment loop, inspired by Karpathy's autoresearch. The core workflow is Evaluate → Improve → Test → Human Confirm → Keep or Rollback, which includes:

• Dual Evaluation System: An 8-dimension weighted scoring rubric (total 100 points), where 6 structural dimensions (e.g., Frontmatter Quality, Instruction Specificity) analyze SKILL.md statically; 2 effect dimensions (Overall Architecture, Test Performance) use sub-agents to execute test prompts and compare output quality with and without the skill.
• Ratchet Mechanism: The optimization loop retains only improvements with a total score strictly higher than the baseline, automatically rolling back regressions to ensure monotonic improvement.
• Human-in-the-Loop: After each optimization step, the process pauses for user confirmation of test prompt design and scoring results to avoid automation bias.
• Exploratory Rewrite: When hill-climbing optimization stalls continuously, an exploratory rewrite can be proposed to escape local optima, but requires explicit user consent.

Optimization strategies are prioritized (P0-P3), addressing issues from performance to readability, with each round modifying only one dimension.

Applicability and Constraints

The Darwin skill is specifically designed for optimizing AI Agent SKILL.md files, adhering to the following constraints:

• No Core Function Changes: Only improve "how it's written" and "how it runs," not "what it does."
• No New Dependencies: Avoids adding scripts or resource files not originally present in the skill.
• Single-Dimension Changes: Each round's modifications are limited to one dimension to ensure attribution.
• File Size Control: Optimized SKILL.md must not exceed 150% of the original size.
• Rollback Capability: All changes are made on git branches, supporting git revert.
  This skill is suited for scenarios requiring systematic iterative optimization of AI Agent skills, especially for teams emphasizing empirical validation and version management.