I’m looking at running a quantized DeepSeek model locally and trying to decide whether a 4-bit version is good enough for tasks that need several steps of reasoning. Most comparisons I’ve found focus on speed or memory use, but I’m more interested in the kinds of mistakes that show up after quantization.
Has anyone compared the same model and prompts at different precisions on tasks with intermediate calculations or constraints? I’d especially like to know whether 4-bit quantization changes answer quality consistently, or whether the impact depends more on the task and prompt.
Can you answer this question?
Write Answer0 Answers