oh-my-knowledge

oh-my-knowledge addresses a common but often invisible problem in AI applications: when prompts, skill documents, RAG recipes, or workflow artifacts change, how do you know whether the change actually improves the system? Many teams rely on intuition, a few demo conversations, or anecdotal success, which makes release decisions hard to explain, reproduce, or defend.

OMK treats the knowledge artifact itself as the variable under test. It compares versions using the same model and evaluation samples, producing evidence-backed verdicts with confidence intervals, failed samples, cost, and ship recommendations. It can also audit whether an artifact is coherent and measurable, visualize scores and diagnostics, and observe real task trajectories to surface production gaps. Those gaps can then become regression samples, creating a closed loop between evaluation, observation, and improvement. Inside DeepSeek Harness, it can reuse the current profile and open persisted task trajectories in Studio.

It is best suited for authors and maintainers of prompts, RAG systems, agent skills, and knowledge-driven workflows who need trustworthy release decisions. If you frequently need to answer why one version of a knowledge artifact should be accepted over another, oh-my-knowledge helps turn that decision into a repeatable, evidence-based process.