Model wrote the answer before it finished reasoning
Kimi K 2.5 I turned on reasoning and websearch (It didn’t do any).
Model started reasoning, gave the answer. Then in the same space where it reasoned it continued (it didn’t spin new reasoning), while at the same time reasoning in another place, finishing it and writing ad nauseam the response under that finished reasoning. All of hat got deleted (which is why screenshot number 4 is just approximation of how it looked like compared to screenshot 4). At the end number of tokens NaN. That was after I asked if “Message is empty” after reasoning is normal (previously model reasoned and didn’t give an answer). Looking at screenshots they should be looked at from the bottom up.
I don’t mind the cost. I’m a power user. I’m just scared that the accuracy won’t be good.
Log in to comment and vote
Comments9
Mark Florkowski
Can you please provide the id (from the URL) of the chat where you encountered this behaviour? It seems like there may be a bad provider in the pool for that model that is not handling special tokens/reasoning correctly
Sapphire Crab
This:
https://t3.chat/chat/9e744fb8-3116-41bf-89f7-f4ef00ae0085
?
You can view my whole chat history. I don’t talk super secret stuff on cloud models.
Sapphire Crab
https://t3.chat/chat/6602c7b9-fac7-4a6a-941b-0f23523bc024
https://t3.chat/chat/0b67b69c-b96a-4e39-9600-759cb729b9d1
I tried reproduce problems with Kimi K 2.5, but she wouldn’t reason twice in the same response. Duplification of screenshots is there. Given that on Stack Overflow years ago I was treated like trash and my previous bug has been “pending” for over a month, I am not sure if you will do anything. Here is hoping. But I had to revive my twitter and twice blast theo for this. Which is not how it should work.
Mark Florkowski
Thank you for these. I have gotten the providers from the logs, and am checking with OpenRouter as well as trying to repro this behaviour on my end.
—
I did not even know about the twitter thing, so that had nothing to do with me responding here or not.
I responded to your post here (posted at nearly 1am my time) within 12 hours of it being posted, which, given the fact I am a single person managing all incoming support requests, I think is fairly reasonable.
Sorry about your old request, sometimes things do slip through the cracks.
Sapphire Crab
I am simply used to being treated badly by programmers. I wonder if I should open a bug report of multiplying attached files (especially photos) that I managed to replicate few times.
26 years ago I tried to go to Linux - people told me that without knowing enough programming to code my own driver I should stop.
I had over 10 years gap with Turbo Pascal, so I wanted to pick C/C++ and people recommended this site “Stack Overflow”. From what I hear I think you suspect what kinds of responses I got. So I didn’t learn any.
And then it felt odd. My bug reports stayed “pending”, my comments stay “pending”. I manage to discover 3 bugs by accident and reproduce them in last 48 hours. But I decided against it, because it would be “pending”, besides I already felt like a…horrible person for going on Twitter. If Theo didn’t suggest in his video that this kind of leverage is valid, I wouldn’t use it. And I feel dirty doing that regardless.
I take care of 2 disabled people - family members. Often 60+ hours without sleep. I kept watch over my dad as I tried to trouble shoot from my end from 1 am to 8 am. I am Polish and while I doubt your surname indicated where you live, you at least know where time wise I tried to help. Because it’s mostly between 1 am and 8 am, because then I have “free time”.
I comment on other issues and my comments are still “pending”. I feel like pariah. Should I feel some form so that Admin doesn’t have to waste their precious time to check my every message if it’s not hate speech or something? Like 12-18 hours back I made a comment on how the thread about how nice it would be to have usage meter when you’re with the model. And I explained to people how difficult it might be and what are the best practices to not use as many tokens…
Pending.
If I should write less and stop being pain in your butt (even though I didn’t plan to be), just say so and I will shut up.
With Regards,
Jan Negrey,
Kraków Poland.
Mark Florkowski
You should submit things as you find them, and not worry too much about whether they are “pending” I will see them, and I will get to prioritizing them.
Sapphire Crab
You’ve been kind to me, which is rare. I already submitted a minor UI bug and I will hunt for bigger one that I had issues with few days ago.
People generally hate me almost as much as I hate myself. So it’s nice to not be talked down to, but spoken to like an equal (maybe a bit annoying one, but not egregiously so).
Mark Florkowski
We’re all just people. I want to build a good product, so I appreciate you taking the time to point out some of the rough edges :)
Sapphire Crab
I forgot. Before that happened (which felt like at least 2 auto regressive models writing over itself), there was one more thing.
MY message started with: “To be exact Sicarius…”
In one screenshot you can see what model put out if you collapse reasoning. But I will show you reasoning.
Reply “message is empty” does raise some flags. That was 2 prompts before that problem above. I know such bugs don’t interest you, so I guess I will shut up.