Skill Evaluation Pro

Problem

When evaluating local Skill behavior, the hard part is not whether a run succeeds, but how to compare candidates consistently: different models, frameworks, and multiple Skill variants can produce unstable results if evaluation sets, drivers, and judging criteria are assembled by hand. Skill Evaluation Pro turns a natural-language evaluation intent into a controlled workflow, supporting single-Skill validation, multi-Skill comparison, multi-model comparison, and runtime-framework comparison.

How it works

The process advances conversationally. First, it confirms the evaluation scenario; next, it selects the target Skill, driver model, and judge model; then it uploads an existing evaluation set or generates one automatically; finally, it presents a task summary and scoring report. Key steps include:

• Scenario confirmation: asks only for the scenario before introducing model or dataset choices.
• Configuration: selects the Skill, driver model, and judge model, then confirms the setup.
• Data preparation: uploads a set or generates data, then previews it for confirmation.
• Execution and reporting: handles auth, packaging, upload, and polling silently, then surfaces the result summary.

Boundaries

This skill is for automated effect evaluation of local Skills. It does not replace code review, system performance stress testing, or model training/fine-tuning. Interaction stays in Chinese, uses numbered tables and concise summaries, and avoids exposing internal commands, fields, or IDs. If the goal includes stress metrics or training pipelines, use a separate workflow.