Retrieval-Augmented Generation chatbot, part 2 - LangChain, Hugging Face, OpenSearch, AWS

We'll walk you through the creation of a Retrieval-Augmented Generation (RAG) chatbot using open-source tools and AWS services like LangChain, Hugging Face, Amazon SageMaker, and Amazon OpenSearch Serverless.
Part 1: • Retrieval-Augmented Ge... - LangChain, Hugging Face, FAISS, Amazon SageMaker, and Amazon TextTract.
⭐️⭐️⭐️ Don't forget to subscribe to be notified of future videos. Follow me on Medium at / julsimon or Substack at julsimon.subst.... ⭐️⭐️⭐️
We start by deploying Mistral 7B, a cutting-edge open-source LLM, onto a SageMaker endpoint. Following this, we work with the Reuters dataset, a Hugging Face dataset comprising 20,000 news articles. We break down these articles into smaller sections and apply bge-small, a compact open-source embedding model, to them.
Next, we proceed to index these sections into an Amazon OpenSearch Serverless vector index, which we then query through LangChain.
Additionally, aside from the RAG demonstration, we delve into some vital yet often overlooked steps related to authentication and security for OpenSearch Serverless.
- Notebook: gitlab.com/jul...
- LangChain: www.langchain....
- Amazon OpenSearch Serverless: docs.aws.amazo...
- Embedding leaderboard: huggingface.co...
- Embedding model: huggingface.co...
- LLM: huggingface.co...

[@bhanuchirutha]

great , I agree sometimes you have to spend a lot of time on IAM than the original problem what a mess

[@juliensimonfr]

Yes, even if you know what you're doing, it's difficult to be 100% sure 🤣

[@pfunnell]

this is great, my son and I have both been working on something similar, each for different use cases, this is going to help both of us, salut!

[@juliensimonfr]

Glad I could help!

[@Martyniqo]

Thanks a lot!

[@juliensimonfr]

You're welcome!

[@mtin79]

Merci beaucoup! Very helpful 👍🏻

[@juliensimonfr]

You're welcome!

[@ravindranshanmugam782]

Excellent to bring out IAM issue, I am experiencing it. I am runnig program in Jy.notebook, not ec2. As admin user, how can I attach role (aoss) to achieve this. Thanks.

[@juliensimonfr]

If you work on your local machine, you need AWS credentials (~/.aws/credentials) with enough permissions, and you need to set all roles explicitely with the arn, e.g. you can't use get_execution_role() in the SageMaker SDK. See ruclips.net/video/K3ngZKF31mc/видео.html

[@Ben-gp5ty]

Julien, if we have a document in S3 that when deleted, i want to trigger a lambda to delete the chunks and embeddings in opensearch belonging to this document. How do I do so ?

[@juliensimonfr]

Each chunk should have metadata on the source document, which you could use to query and delete.

[@TheMrGoodkind]

This is really great! Thank you! If I want to add this RAG-augmented chatbot to my personal website, how would I do that?

[@WagnerHeleno]

Hi Julien, your video is excelente. I have a question, with this solution (using opensearch service) is possible to deploy thought Lambda Service too?

[@juliensimonfr]

Hi, serverless inference on AWS is interesting, but no GPUs...

[@XShollaj]

Thank you Julien! Will there be a tutorial deploying this in a front end chat interface ?

[@juliensimonfr]

no, I couldn't write UI code to save my life ;) Gradio has a chatbot interface, this would probably be a good place to start www.gradio.app/docs/chatbot

[@XShollaj]

@@juliensimonfr Thank you! Highest standards for tutorials as always!

[@sergioquintero4624]

Hi. Can you explain a little more about the cost of this PoC ? Thanks

[@juliensimonfr]

Check the pricing for the AWS services involved :)

[@caiyu538]

thumb up first and then watch.