I’m trying to make a small evaluation set where I can compare prompt changes without output randomness muddying the results. If I set a seed in an OpenAI API request, how consistent should I expect the response to be across repeated calls?

I’m especially curious whether the same seed still helps when requests are sent at different times, or when a model alias may point to a newer snapshot. Is it reliable enough for regression tests, or should I treat it as a best-effort setting and allow for some variation?