Skip to main content

custom maximum context window

Feature Request

context windows at the moment are designed for focused conversations, preserving as much context as possible at severe cost to usage limits for us. this makes it extremely costly to have a single chat window open for varied questioning where context hardly applies, at the moment you are burdened to create a ton of new windows to keep new questions cost low or suffer intense and increasing token usage. if we were able to set more aggressively small windows we could keep costs down and maintain usability for that use case.

Status: Gathering Interest2 comments

Log in to comment and vote

Comments2

  • Mark Florkowski

    Team

    starting a new chat when the topic changes also accomplishes this?

    • Indigo Magnetosphere

      its a huge pain to do repeatedly, there is a reason this feature would not be unique to t3 chat. if you use any of the 2-3 cost models or 0/1 with any grounding the token burn is often bad enough you have to make a new tab every second or third question. this usage pattern where you ask many context irrelevant questions rapidly is definitely not representative of many t3 chat users, but for me its providing enough issues i made this feedback post about it.