I’m troubleshooting a locally hosted DeepSeek-derived instruction model. Sometimes it answers my question and then continues with a new user message that I never wrote. In other cases, role markers appear in the output.
Before blaming the model’s instruction following, I want to check whether my inference setup is formatting the conversation incorrectly. The exact checkpoint and its tokenizer files seem relevant here; I don’t want to assume every DeepSeek-derived model uses the same format.
What should I inspect first?
- Whether the server uses the chat template shipped with the checkpoint.
- Whether my client adds role markers that the server also inserts.
- Whether the rendered prompt ends at the expected assistant-generation boundary.
- Whether the runtime recognizes the checkpoint’s end-of-turn tokens.
Is that a sensible order for chat template debugging, or would you start elsewhere? I’m especially interested in how to distinguish an incorrectly formatted prompt from a stopping issue: both seem capable of producing an answer followed by an invented conversation.
I can share the checkpoint identifier, runtime version, and a sanitized rendered prompt. Which of those is most useful for diagnosing this without posting an entire server log?
Can you answer this question?
Write Answer0 Answers