Luban: AI Agent Skill Optimizer

The Problem: Ad-Hoc Skill Refinement Lacks Engineering Rigor

In the AI Agent ecosystem, developers often struggle with inconsistent skill quality. A skill might be poorly structured, have ambiguous steps, lack error handling, or rely on ad-hoc, inspiration-driven tweaks for "improvement." This leads to non-reproducible results, unquantifiable gains, and a high risk of introducing regressions or new flaws. The core challenge is transforming the optimization of skill texts (like SKILL.md) from an unstructured "art" into a systematic engineering process that is measurable, rollback-capable, and governed by safety guardrails.

How Luban Works: A Layered Architecture for Closed-Loop Optimization

Luban.skill is a meta-skill designed to automatically elevate the quality of other skills. It operates on a rigorous evaluation and iteration framework:

Core Capabilities and Workflow:
1. Ten-Dimension Evaluation Rubric: It avoids subjective assessment by using a Rubric with 10 weighted dimensions (e.g., Workflow Clarity, Failure Mode Encoding, Live Performance). Each dimension has clear scoring rules (deterministic/LLM-based), ensuring objectivity and explainability.
2. Dual-Mode Optimization Engine: It offers two modes based on need or initial score:
- Quick (Lightweight Mode): Runs 3 rounds of self-refinement (Self-Refine), using .bak files for rollback. Ideal for swift fixes.
- Full (Comprehensive Mode): Runs 5 rounds of iteration, employing git branches for version control and independent multi-judges (sub-agents) for scoring. It concludes with an optimization dashboard, suitable for deep, high-stakes refinements.
3. Structured Optimization Phases: Optimization follows a strict Phase 0-3 pipeline—from diagnosis, editing, evaluation, to review—with enforced constraints. For example, it modifies only one dimension per round to avoid confounding variables and mandates the creation of test prompts (test-prompts.json) to validate real-world performance.
4. Key Safety & Governance: It implements the "Five Design Pillars," such as "Ratchet Rollback" (ensuring each iteration only keeps net-positive changes) and "Human-in-the-Loop" (requiring confirmation for each optimized skill). Architecturally, it divides governance into layers from L0 (atomic operations) to L4 (meta-cognitive audits), ensuring robustness.

Applicability and Important Caveats

Luban.skill targets structured text-based skill files (e.g., SKILL.md). Its methodology is deeply tied to the predefined Rubric and strategy library (P0-P3). Therefore:

• It optimizes "how it’s written," not "what it does": It will not alter a skill’s core functionality or intent, only its expression, structural completeness, and executability.
• Dependent on a Specific Tech Stack: The workflow is tightly integrated with git for version control and TSV/JSON data files, requiring a compatible runtime environment.
• Human Review is Recommended: Even outputs from the "automatic" Full mode should undergo human review gatekeeping, especially for high-risk modifications.
• Not a Universal Fix: For extremely disorganized skills or those with fundamental logical flaws, it may recommend a full rewrite rather than incremental patching. Its optimization ceiling is bound by the original design of the SKILL.md.