They are already included with every message, as the full history is sent with each message.
Chocolate Melon
I don’t mean at the top of the history, but at the new iteration. For longer chats, many models struggle with instructions that are all the way at the top in the system prompt, which can reduce accuracy. Just for context:
Ah I see. That will definitely increase costs, potentially significantly due to lack of caching, but i guess could be better for getting models to comply. Will experiment
Log in to comment and vote
Comments3
Mark Florkowski
They are already included with every message, as the full history is sent with each message.
Chocolate Melon
I don’t mean at the top of the history, but at the new iteration. For longer chats, many models struggle with instructions that are all the way at the top in the system prompt, which can reduce accuracy.
Just for context:
https://youtu.be/Wvj1mTqyzsQ?si=fDeAM-jImcsY9pMA&t=1394
Mark Florkowski
Ah I see. That will definitely increase costs, potentially significantly due to lack of caching, but i guess could be better for getting models to comply. Will experiment