I’m considering running a DeepSeek R1 distilled model on my own machine, but the model sizes and hardware requirements are hard to compare. For someone choosing between a desktop GPU and a CPU-only setup, what should they check first?
I’m mainly wondering how much system RAM or GPU memory different model sizes need for practical inference, and whether a smaller distilled model is a sensible starting point for local use.
Can you answer this question?
Write Answer0 Answers