I’m trying to understand OpenAI prompt caching before using it in our cost estimates. Our support assistant sends the same instructions and tool definitions on most requests, but the cached-token count varies more than I expected.
The customer’s message changes each time. We also include a request timestamp near the beginning of the prompt, and our application sometimes changes the order of the tool definitions when building a request.
What should I keep constant for a useful comparison?
- Does a changing timestamp near the start affect caching for the instructions that follow it?
- Does tool-definition order matter even when the available tools are identical?
- How should I separate prompt differences from timing or model-specific caching requirements?
I’d like to run a small, repeatable test using the reported cached input tokens, rather than guessing from response speed. Which request details and usage fields would you log to explain the differences?
Can you answer this question?
Write Answer0 Answers