The lab
If you finish early: score one task
One task teaches a new ability: turning your taste into a written standard, and checking whether an AI can judge by it. Pick the task where your win was closest.
- Write a scoring guide. A scoring guide is a few lines on what a great answer includes, what earns top marks, and what is an automatic fail. It turns a wish into lines anyone can check. "Sounds like me" becomes: no sentence over 25 words, opens with a concrete example, no hedging phrases. Use Prompt 5 in Part 3 with your one task. When it offers more tasks, say one is enough for now. Done: a scoring guide that says what earns each score from 1 to 5.
- Score the hidden answers yourself, from 1 to 5. Done: your scores, written down before you see anyone else's.
- Bring in an AI judge. An AI judge is a second AI, from a company that made none of your options. It scores the same hidden answers against your guide. Use Prompt 6 in Part 3, in a fresh chat. Done: two sets of scores, and one line on where you and the judge differed.
A disagreement usually has one of three causes. Your guide is missing something you care about. The judge misread. Or length and polish swayed you.