I’m adding an AI feature to an app and want to compare prompt or model changes without relying on a handful of memorable examples. For an OpenAI API workflow, how do you build a small evaluation set that actually reflects what users will ask?

I’m especially curious about balancing ordinary requests with edge cases, and how often people update their test examples as the product changes. Do you keep the evaluation data separate from production conversations?