I’m making a small change to a prompt used by an OpenAI model, and a few hand-picked examples make the new version look better. I’m not sure that tells me much: the examples are probably too similar to the requests I had in mind while editing.

For people who test prompts in production, how do you put together a prompt evaluation set that catches regressions as well as improvements? Do you keep expected answers, score specific behaviors, or compare outputs side by side? I’d especially like to hear what has worked for changes that affect tone or instruction-following rather than factual accuracy.