I’m trying to estimate whether a local DeepSeek deployment will fit on my available GPUs, but model size alone doesn’t seem like enough to plan around. A long context and several simultaneous requests can add a lot of KV cache memory, and different serving runtimes seem to report different usable context limits.
What inputs do you use for a realistic VRAM estimate—model variant, context length, concurrency, runtime overhead? I’m looking for a practical way to size a deployment before spending time tuning it.
Can you answer this question?
Write Answer0 Answers