I’m testing a long, mostly fixed system prompt with the OpenAI API, followed by a different user message each time. I’m trying to work out which edits to that shared prefix make the prompt cache miss: for example, adding a timestamp, changing a small instruction, or reordering sections.

Has anyone measured this in a real application? I’d be interested in what you log to tell a genuine cache hit from normal variation in response time.