I’m trying to estimate the value of prompt caching in the DeepSeek API for a workload where every request shares a long system prompt, but the user message changes each time. I’m not sure how much of the prompt has to match for the cache to help, or whether small changes near the beginning invalidate the useful part.
Has anyone measured the effect on latency or cost with a real request pattern? I’d also like to know whether prompt caching behaves consistently enough to factor into an application’s normal performance estimates.
Can you answer this question?
Write Answer0 Answers