I’m trying to work out whether prompt caching is making a meaningful difference to my API spend, rather than just assuming it is because my requests share a long system prompt.
When comparing usage across calls, which cached token usage fields or billing details should I look at? I’m also curious how much the prefix needs to match for caching to help in a real application.
Can you answer this question?
Write Answer0 Answers