Response Cutoff/unfulfilled message
Sometimes, when prompted for reasoning models like Claude sonet 4.6, due to network issue or focus change, streaming response is raised. After refreshing the page, some of the response/reasoning is displayed, but the rest of it is not. Sometimes the whole message is empty, sometimes it shows empty message but shows reasoning. We can re-run the prompt, but the tokens are all ready deducted for the previous request, and re-ruining the request will drain more tokens. For costly models like sonet 4.6, it could cost approximately the whole 4 hour limit.
I think after firing the prompt, the response, wether streaming to client (active) or not (broken due to network issue, inactivity), should always get back the model response after reload/refresh. I have observed it 5-6 times as I prompt the AI and go to do other things.
Attaching a latest screenshot of this happening to me in between streaming response.
Log in to comment and vote
Comments4
Mark Florkowski
This typically happens when the model uses it’s entire token budget in reasoning, not because we are not persisting responses from the model. Responses are persisted regardless of whether you are viewing the page.
What is probably actually happening here is that if the model is taking long enough to respond that you leave the page and come back, it is in a state where it is going to overthink, not that the response is not saved.
—
If you have an example url of a chat thread where you encountered this issue I am happy to verify in the logs.
The issue of models using all of their token budget in reasoning typically occurs in 3 main cases:
Too high of a reasoning level is selected - model goes into loops, overthinking, wastes budget.
Prompt is not specific enough - model does not have enough information to go off of, so it tries to follow too many paths and wastes all of it’s budget.
Prompt is asking for too many things at once - when the model needs to plan out a response to multiple questions at once, it can run out where it wouldn’t have if you broke the prompt into multiple.
Coffee Syrup
I will check if I have any threads active where I have encountered the issue. As far as I remember, this has only happened on Claude sonet 4.5/4.6. Also, I give it a detailed long message, so maybe the tokens gets used due to long message and asking for detailed input. I will try to find older threads or reproduce the same error.
Coffee Syrup
I tried to repeat the same prompt. Got the same empty message response. So I think it’s because the token budget is all used up in reasoning as you stated.
Maybe if possible, you can state or indicate the token budget is reached or exceeded.
Mark Florkowski
I think we could do somthiing like that, will think on the best way to visualize it 👍