I have an API request that includes the same long instructions and reference material for most users, followed by a small amount of user-specific input. Has prompt caching made a noticeable difference for this kind of workload?

I’m wondering whether it’s worth restructuring the request to keep a shared prompt prefix, or whether the savings and latency improvements are usually too small to justify the extra care. If you’ve tried this in production, what did you measure?