A quantized DeepSeek model loads successfully on my GPU and handles short prompts. With longer conversations, memory usage climbs until the runtime fails or shifts work to system RAM.

I clearly budgeted for the model file rather than the whole workload. What should I measure to estimate KV cache memory usage for the specific model and inference engine I’m running?

I can share the model identifier, quantization, GPU VRAM, runtime version, configured context limit, and actual input length. Are there other settings that matter just as much, particularly cache precision or whether the engine reserves memory in advance?

One user now, several later

I also want to understand concurrent inference memory requirements. If one long conversation works, that doesn’t tell me whether two simultaneous requests will fit. Would you test prompt length and concurrency separately first, or is there a better way to establish a safe operating limit?