I’m considering serving a DeepSeek model locally for a small team, with several people sending requests throughout the day. Most examples I’ve found focus on getting a model to run at all, rather than what happens once requests compete for the same GPU.

For people running several users on one local GPU, what has mattered most for usable throughput: model size, quantization, batching, or the serving stack? I’m trying to understand what to measure before choosing a setup, especially when response times matter as much as peak tokens per second.