I’m adding a Stop button to an app that streams responses from the OpenAI API. Stopping the text on screen is straightforward, but I’m not sure what needs to happen behind it.
The browser connects to my backend, and the backend opens the request to OpenAI. If I abort the browser request, the upstream connection might still be running unless my server explicitly closes it.
For OpenAI streaming cancellation, I’m trying to separate three things:
- Stopping updates in the interface.
- Stopping the backend from receiving more output.
- Stopping generation on the provider’s side.
Does closing the upstream connection reliably accomplish the third, or does cancellation depend on the endpoint and request mode? I also need to understand token usage after cancellation: if the stream ends before I receive final usage information, how should I reconcile that request with the usage recorded for billing?
I’d appreciate an explanation that distinguishes ordinary streaming requests from background responses rather than assuming they behave identically.
Can you answer this question?
Write Answer0 Answers