The empty message and the wasted tokens
Today I wanted to generate the document and I am sure it not only did not do anything after processing for 10-15 minutes, but it wasted (took away) my tokens. I had like 86%, generated one successful document with about 6-8% (maximum), another two with another 10-12% (combined)… and I have only 52% now. Simple math tells me the token of 12-15% was deducted from my account after I did not get the output (genated document).
https://t3.chat/chat/43e45e0c-caa0-430a-9a08-26d5785e2676
Please review my account and if needed - restore the missing tokens. Thank you!
Log in to comment and vote
Comments3
Mark Florkowski
It appears that the model exhausted it’s token budget for that response during reasoning. You should see a section in the response message labelled “reasoning” which you can expand to see that the model did respond.
as a courtesy, I have refunded some credit to your account, but be aware that this is a thing that can happen with prompts that are too broad or when the reasoning level is set higher than necessary. The model may begin to “overthink” and be unable to complete it’s response in a reasonable amount of tokens. You can read a bit more about this here:
https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices#overthinking-and-excessive-thoroughness
The article is talking about Opus in this case, but the same concepts apply fairly universally with any thinking model.
Silver Milkshake
Thank you for the quick response and refund of some tokens.
Previously when I used Claude Sonnet it was working pretty well. At some cases it was working extremally well and even if some of the tasks cost a way more tokens than I expected the final result was worth the price.
Recently I am scared to use any of the tools as they freeze that often that it results in two unexpected and undisered results: the model is working for 10 minutes and spew nothing and the model simply steal my tokens. I know it may sound absurd, but this is what I experience.
What could be the possible solution? Shall I use the old models at medium or low effort and specify super narrow and extremally limited tasks (prompts?).
Is there any way to have the model to STOP without losing (wasting) tons on time, energy and tokes… and tell the user to refine the specific prompt? Or at least to show me the signal (visual cue?) to know I should interrput it before it takes away most of my tokens?
And the last question - if I bought the higher sub plan - would this problem disappear or would it simply took a way more tokens with the same effect? I am pretty upset and frustrated as nowadays it seems most of my tasks cannot be generated either I do not know how to stop wasting the resources as I do not know when the model gets into the overthinking mode 😭
Mark Florkowski
This is certainly an approach you can take. If the older models were working better for you, there is no reason to switch to the newer models that do not work as well for you. It is generally advisable to start with lower thinking levels and only increase if you are not getting the quality of responses that you expect.
Nt intrinsically, no. You can add to your custom instructions in settings something to have the model ask you to provide more clarification if it believes the given prompt is too broad/does not have enough information to complete in a reasonable time. We have a little bit of this in the system prompt already, but we try to keep it relatively unbiased so that you as a user can have more control through system instructions, as everyone has their own preferences for response style/
The higher tier plan has higher limits all around, both in the monthly/4-hour limits, as well as the max allowed length/cost per message.
To be entirely honest with you however, you amy be better suited with a first party Antrhopic subscription depending on what you are actually trying to do, as they are much larger/more funded, and own the models, so they are able to subsidize your usage in a way that we as a small company that must pay sticker-price for access simply cannot.