https://blog.baifan.site/en/tags/ab-testing/Recent content in AB Testing on ByF (BaiFan) · Daily Deep ThinkHugo -- gohugo.ioenblog@baifan.site (ByF (BaiFan))blog@baifan.site (ByF (BaiFan))CC BY-NC 4.0Fri, 09 Oct 2026 14:48:00 +0800https://blog.baifan.site/en/evaluating-agent-skills-first-principles/Thu, 08 Oct 2026 18:18:00 +0800blog@baifan.site (ByF (BaiFan))https://blog.baifan.site/en/evaluating-agent-skills-first-principles/<p>By the time I was maintaining the fifteenth or so skill in <code>dev-skills</code>, I realized I had no reliable way to tell whether any of them actually worked. Every demo produced clean output. But the output had no control group, no causal grounding — I knew what happened with the skill mounted, not what would have happened without it.</p> <!-- more --> <h2 class="relative group">How I Was Judging at the Start <div id="how-i-was-judging-at-the-start" class="anchor"></div> <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"> <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#how-i-was-judging-at-the-start" aria-label="Anchor">#</a> </span> </h2> <p>The original method was simple: write a skill, load it into Claude Code, run a task, check whether the output looked right. The first version of <code>grill</code> passed this way — I fed it a service design with a concurrency bug, the agent flagged a lock contention issue, I marked it valid.</p>