I’m adjusting a prompt for a support assistant, and the new version sounds better in a few hand-picked tests. That’s not much evidence, though: it may have improved one kind of answer while making another less reliable.

How do you build a small evaluation set that reflects real user requests without turning the process into a big testing project? I’m especially interested in ways to compare OpenAI model outputs when answers can be phrased differently but still be equally useful.

Do you rely on human reviews, a scoring rubric, another model as a judge, or some combination? Curious what has worked for teams shipping prompt updates regularly.