My local DeepSeek setup feels responsive when I test it alone, but I expect a few people to use it at the same time. A single-prompt tokens-per-second figure doesn’t tell me much about what happens once requests overlap.
For a useful comparison, which numbers do you track under concurrent load—time to first token, overall latency, throughput, or something else? I’d also like to know how people keep prompt lengths and hardware settings consistent between test runs.
Can you answer this question?
Write Answer0 Answers