Summarise chat once token length has been exceeded
Similar to https://feedback.t3.chat/p/summarize-chat-feature
Refreshed T3 Chat today (10/02/2025) and got hit with a This thread has exceeded the token limit. Switch to a model with higher limits.
Which is understandable with the amount of things I was throwing at o3-mini high
But now if I want to start a new chat I lose all context and need to start from scratch
I tried generating a summary of the chat using Gemini 2.0 Flash but that just failed Stream failed for unknown reasons
My suggestion would be to take my chat, split it into manageable chunks for either o3 mini or Gemini to summarise and give me that output so I can take it to another chat
Log in to comment and vote
Comments14
Mark Florkowski
Definitely something we are looking into, as well as the option to truncate the oldest messages from the context
Aquamarine Teacup
The option to truncate out-of-context messages would be really excellent
Apricot Gizmo
Most important feature would be to get a heads up, “context window almost full, you should ask for a summary to use as a prompt in a new chat”
Pink Storm
I have faced the same issue and it has forced me out of my preferred approach which is single thread per app.
I can think of different levels of solutions for this
1. Easy, good enough to unblock:
Allow switching model to something like gemini for me to manually retrieve a summary of what has happened so far to seed a new chat.
2. Truncate oldest messages from chat:
Allows seamless continuous conversation but has a downside that important user instructions might be truncated out. E.g. User asks at the beginning of conversation: respond with code snippets rather than rewriting my whole code files.
3. Rolling context:
Similar to rolling average. Whenever limit is reached, prompt the model (or a cheaper model) to summarize the 90% oldest messages in the conversation and to extract any long term user requirements.
Replace the 90% oldest messages with the generated summary + user instructions, append the 10% most recent messages and carry on the chat.
The UI can show the full message history and does not have to follow truncating/summarizing.
These are my thoughts on the matter and I thought I would share them for your kind consideration.
Orange Pond
I would love to have possibility to choose from those options when the chat exceeds the context length, so I can decide based on my needs
Beige Antelope
I just came to report this too. It’s really rough not being able to continue a conversation after a certain point and losing all context. I don’t really care about the code snippets from 20 messages back, but having to essentially “pay again” to generate the context by messaging claude before asking my next thing really sucks. Also needlessly increases Theo’s claude bill!
Beige Antelope
Additionally, I’d suggest switching this from “Feature Request” over to “Bug Report”
Olive Penguin
It doesn’t have to be the same model. Even something like 2.0 Flash can do a good summary. Theo just needs to create a good UX with it, and we will be golden!
Magenta Andromeda
Inspiration for implementing this is Claude Code. It shows what percentage of context window remains before summarization kicks off. It’s pretty cool.
Jade Elephant
Merged request
Summarize Chat feature
This is probably a bit meta, but I would love to be able to submit an entire chat (or reference a chat?) into t3.chat that an LLM can then summarize the discussion for me, highlighting key pieces of information, decisions etc. This is particularly useful when exporting as I can then take the summarized chat, feed that into something like Cursor’s Compose or Agentic workflows, and have them perform work that I have planned out with the help of T3’s LLMs.
Another alternative to this could be some way of condensing chat information, decisions, etc into a format easily readable by cursor (I have read that LLMs tend to perform better reading in XML apparently rather than raw text?).
Jade Satellite
you can just do it yourself by asking please summarize our chat from the beginning.
Beige Cascade
I would love this as a feature, I’m often hitting the context limit
Purple Apple
the option to automatically summarise / compact (maybe with a second model?) and remain within a defined percentage range of context would be nice (maybe a min-max percentage slider, though that might be jank af in reality). I’m seeing good results using models like this in big, old, ugly, messy codebases using open code. Maybe it’ll help squeeze more out of these models
Red Pineapple
Think a way of summarising is important especially with the yellow alert that’s appearing lately