I send several OpenAI API requests with the same system instructions and tool definitions, but each request also has some dynamic user text near the beginning. I’m trying to work out whether changing that early text prevents the later, repeated part of the prompt from being cached.
Has anyone tested how sensitive prompt caching is to changes in prompt order? I’d be interested in what you saw in the usage details, especially with a long shared prefix.
Can you answer this question?
Write Answer0 Answers