I’m considering running a DeepSeek model locally, but the context length listed for a model doesn’t tell me much about the hardware I’ll actually need. I assume a long context can use a lot of memory, but I’m unsure how much depends on the model size, inference settings, and the amount of text in a request.
For anyone running DeepSeek on a consumer GPU, what context lengths have been practical for you, and where did memory use become the limiting factor? I’m trying to plan a setup before buying hardware rather than relying on the advertised maximum.
Can you answer this question?
Write Answer0 Answers