Early this week, Microsoft shared its learnings from the preview launch of the new Bing experience. Microsoft confirmed that in long, extended chat sessions of 15 or more questions, Bing Chat can become repetitive or be prompted/provoked to give responses that are not necessarily helpful or in line with its designed tone. This is because of following reasons:
- Very long chat sessions can confuse the model on what questions it is answering.
- The model at times tries to respond or reflect in the tone in which it is being asked to provide responses that can lead to a style we didn’t intend.
In response to the above issue, Microsoft last night around some restrictions around Bing Chat. From now on, the Bing chat experience will be capped at 50 chat turns per day and 5 chat turns per session.
A turn is a conversation exchange which contains both a user question and a reply from Bing.
Microsoft’s user analytics revealed that the vast majority of users find the answers they’re looking for within 5 turns. And only ~1% of chat conversations have 50+ messages.
Once your chat session reaches 5 turns, Bing Chat will now prompt you to start a new topic. At the end of each chat session, context needs to be cleared so the model won’t get confused. You can click on the broom icon to the left of the search box for a fresh start.
Microsoft said that it will explore expanding the caps on chat sessions to further enhance search and discovery experiences in the future.
