I’m trying to use prompt caching for a request that has a long set of shared instructions followed by user-specific details. The shared part stays mostly the same, but a few values and occasional instruction updates can change.
How sensitive is a cache hit to edits near the beginning of the prompt? I’m especially interested in whether there’s a practical way to organize a static prompt prefix without making the request awkward to maintain. If you’ve measured this in an app, what changes affected your cache-hit rate?
Can you answer this question?
Write Answer0 Answers