I’m comparing DeepSeek distilled checkpoints for local inference, and the published parameter count doesn’t tell me what I need to know about memory once the context gets long. A model that loads comfortably for a short prompt can become a different story when the KV cache grows.

For anyone running these models locally, what GPU memory have you found practical for a useful context length, and which model size or serving setup are you using? I’m especially interested in real workloads rather than the minimum needed just to start generation.