I’m considering running a quantized DeepSeek model locally for structured extraction from support tickets. Speed and memory use matter, but a few missed fields could be more costly than slower inference.
For people who have compared quantized and higher-precision versions, what changed on your real workload? I’m wondering which measurements caught quality loss best—exact field matches, valid output rates, or a manual review of borderline examples—and whether the effect differed much between fields.
Can you answer this question?
Write Answer0 Answers