Build An AI Chat Bot with CONVERSATIONAL MEMORY in Spring AI | Spring AI Tutorial

In this video we learn about "How to Build an AI Chat Bot with CONVERSATIONAL MEMORY Using Spring AI"
This is episode two in the playlist on Spring AI Tutorial. In the 1st episode we learnt how to create open ai api key and get started with making basic calls to Open AI using Chat Client APIs. Playlist link given below.
-------------------------------------------------------------------------------------
Playlist : • Spring AI
Code Link : github.com/sha...
-------------------------------------------------------------------------------------
👉🏼 Subscribe us : www.bit.ly/subcodecraft 🚀

[@BenjaminRowley-t1s]

Thanks for the great video! I had previously done something similar and stored the conversation context in a mySql db. I really like how you have used the delimiter. I will try this and see if it improves the chat responses I get. Also your system prompt is really well done.

[@shandilyacodes]

Thanks for your comment and glad that you found the content helpful :) Pls subscribe if not done already! If you like the prompt, today I'll be releasing a video on using Prompts with Spring AI, hopefully you'd like that as well :)

[@backtoGodhead0]

Looks like we still need to keep track of ids and the responses of the LLM and send it with every prompt, doesn't that will blow up the 'context-window' of the model and also the cost repercussions.

[@shandilyacodes]

Thanks for your comment. Yes, sending the ids and LLM response in each conversation indeed does blow up the context window. That's why I suggested one approach where we can use the LLM itself to summarise the discussion so far in short if the context window goes beyond certain size. Other approach is to be track only a few of the last conversations, but downside is LLM could lose track of the previous details, for example, we have had 100 conversations in a thread, but we are tracking only last 25 conversation, so depending on how related things are loss of the previous 75 conversation could lead to uncertain response. Other approach could be create some sort of relation (a directed graph may be?) between the conversation, and use that wisely as needed. So given that LLMs are stateless, idea is to get as creative as possible to get the most optimal result without blowing up the token cost :) Hope you found the video and this comment useful. Thanks!

[@backtoGodhead0]

@@shandilyacodes Thanks for replying back, interesting to hear your thoughts on this. Could you please suggest how the `ChatMemory` object of Spring AI package can help in this regard.

[@Edkahmed]

Thanks for tutorial!! whats the code editor & distro that you're using?

[@shandilyacodes]

Thank you! Glad you liked the tutorial. I am using Intellj Idea with some customisations on a Macbook. Please subscribe if not done already :)

[@cyper0418]

I was wondering why my model lost context when I asked a 2nd question and you saved my day! So, all the history prompt will also be regarded as token_count right? And it will accumulate each time I continue a conversation, if so, that will spend me quite a bit money.

[@cyper0418]

Ah, I realized the prompt history just contains the question part, not that much then

[@shandilyacodes]

Thanks for your comment and I am glad you found the video helpful. To answer your question, yes, all that you send in the conversation context would be counted towards total token consumption and will cost money. One workaround to this is with two combined approaches
1. Keep only a range of last 'few' questions and answers in the discussion for the context. For example, last 7 questions and their answers
2. Summarise the discussion so far so that LLM has the context but less tokens are used
Ofcourse, both the approaches have their own pros and cons, but for some one who is on a budget and wants to play around, these two approaches are good starting point.
If it was helpful, please do consider subscribing my channel, thanks again!

[@bijugeorge6200]

Excellent tutorial.

[@shandilyacodes]

Thanks for your comment. I am glad you liked. If not done already, I request you to please subscribe and help the channel grow :)