I have a backlog of independent prompts to process with DeepSeek and don’t need an immediate response for each one. Does the API provide a batch inference option for this kind of workload, or is it better to send requests through a queue and manage the pacing on my side?
I’m interested in practical trade-offs too: how people handle failed requests, results arriving out of order, and keeping a large run easy to monitor.
Can you answer this question?
Write Answer0 Answers