I’m testing a DeepSeek-R1 distilled model for a small extraction task. The response usually contains the right information, but it sometimes adds a short explanation before the JSON or uses a field name that isn’t in my schema.
For people serving these models locally, can prompting alone make the output dependable enough for a parser, or do you need constrained decoding? I’m also wondering whether results vary noticeably between serving engines such as vLLM and Ollama.
Can you answer this question?
Write Answer0 Answers