Tool calls (such as search) are not retained in the context window in subsequent turns

Steps to reproduce

  1. Ask a model something like “perform some search but don’t show me the queries and the results” [let’s call this “user message 1”]

  2. An underlying LLM provider call happens [input: system prompt + user message 1]

    • If it’s a reasoning model, it reasons for a bit

    • The model may or may not then say something like “Let me search for this:” [let’s call this “response 1.1”]

    • The model writes the tool call

    • The model may or may not then say something like “Alright, tool call executed, waiting for the results now!” (most models don’t)

  3. A web search happens

  4. An underlying LLM provider call happens [input: system prompt + user message 1 + sometimes reasoning + response 1.1 with the tool call + tool call result]

    • If it’s a reasoning model, it will likely reason for a bit

    • The model then says that the search was executed and that it sees the results [let’s call this “response 1.2”]

  5. Ask a model again if it still sees the search results [let’s call this “user message 2”]

  6. An underlying LLM provider call happens [input: system prompt + user message 1 + sometimes reasoning + response 1.1 without the tool call or its results + sometimes reasoning + response 1.2 + user message 2] ← the issue is here

    • If it’s a reasoning model, it reasons for a bit

    • The model then either says that it doesn’t see the tool call itself or its results, or gaslights itself into thinking it forgot to do the tool call in the previous turn (because it no longer sees it) and apologizes for not actually doing the tool call

Model stats

Successfully consistently reproduced with:

  • Claude Sonnet 4.6 (Reasoning)

  • Claude Sonnet 4.6

  • Kimi K2.5 (Thinking)

  • MiniMax M2.5

  • GPT-5.2 (Reasoning)

Successfully (but not consistently) reproduced with:

  • GLM 5 (Thinking)

  • Gemini 3 Flash (Thinking)


“Operating System — Linux” and “Browser — Chrome / Arc / Vivaldi” is simply what I use, this info is unrelated to the issue as it’s almost certainly server-side

Please authenticate to join the conversation.

Upvoters
Status

Gathering Interest

Board
🐛

Bug Reports

Subscribe to post

Get notified by email when there are changes.