Skip to main content

Display Context Window Status / Help User Manage Context Window

Feature Request

It would be helpful to know when the context window is exceeded or about to be reached. The user can then ‘refresh’ context in their next request or optimize their request down to fit in the context window — to hopefully get a better response.

For example, from LM Studio

In the ideal scenario there would be a visual indication of how much of the chat history is in the context window (for example a vertical line that is colored in the area of messages within the context window), but I propose basic feedback would be better than none.

Status: Gathering Interest6 comments

Log in to comment and vote

Comments6

  • Amber Fuelcell

    In the ideal scenario there would be a visual indication

    Ideally, it should just display how many tokens the current chat history is for a particular model. So i don’t have to export chat history, parse json, and manually run a tokenizer to figure it out.

  • Indigo Magnetosphere

    even a little bar indicator to get an idea would be immensely helpful, especially when switching models since they mostly don’t all have the same chat limits.

  • Purple Pickletop

    Not only this, it would be also great to see the context size for each model in the picker.

  • Yellow Toaster

    •

    Merged request

    Indicator of thread length

    It would be really helpful to have an indicator showing how many tokens (or some other indicator) are left before a thread reaches maximum length. That will give some warning so you don’t get stuck with a partially completed output and no way to go forward.

  • Turquoise Asteroid

    Yes! That would be very usefull, l was just about to post a feature request like this

  • Mark Florkowski

    Team

    I really like this idea