RAG with Azure AI Search and Azure Open AI in 9 minutes
HTML-код
- Опубликовано: 22 май 2024
- Equipment used
Blue Yeti Microphone [ www.amazon.in/Blue-Recording-... ]
RAG [ Retrieval-Augmented Generation ] concept explained
📍 How we do we ingest data into our RAG architecture?
📍 How we do we query from our RAG architecture?
📍 Detailed use of Azure AI Search and Azure OpenAI in RAG
📍 Demo of our built in RAG application
Code: github.com/ambarishg/AZURE-AI...
Playlist provided is [ • Generative AI ] 
Наука
Thanks for the video!
Awesome tutorial Ambarish. Keep up the good work.
Great video! Thanks for sharing the code too, greatly appreciated.
Glad to help!
Do we not require any embeddings and vector database here? Also, is vector search not required? If so when is it required? Thank you
Does this work with files with size large than 16mb?
Awesome Video I tried it too wanted to know any way we can do reinforcement learning making the bot train on our feedback
Well done! Brilliant work and very replicable. Thanks a lot!
Many thanks!
Great video, Man!! Thanks for sharing.
Thanks for watching!
Sir I have one doubt, like we are using Azure Cognitive search and the index updates everyday midnight. So we have both new content and old content with us. Now what I want is to retrieve the most recent content first and than old content. How can I implement it?
thanks! very informative video!
Glad it was helpful!
Amazing video! One question, if you have the data source a blob storage, if you delete a pdf file from that blob storage, when you run the indexer is the reference and information about this deleted document deleted from the index as well? thanks!
The data is stored in the Azure AI Search index.
Please note that if we have to remove a document from the index, we have to delete it from the index.
Please also note that in the file upload_docs.ipynb, the function remove_from_index can remove documents from the index
brilliant video!! Much better than the marketing ones from microsoft :)
Wow, thanks!
hey in the section of env where could i find the index name of the ai search
index = "" This is in github.com/ambarishg/AZURE-AI-SEARCH/blob/main/.env.sample
Hi Ambarish, amazing video! Thank you so much for taking the time to make it. Would you be able to help me understand where I can locate the config information? I apologize if some are self-explanatory, I dont even know where to begin on a few of them
searchservice = values_env['searchservice']
index = values_env['index']
searchkey = values_env['searchkey']
category=values_env['category']
#AZURE STORAGE CONFIGURATION
storageaccount = values_env['storageaccount']
container=values_env['container']
storagekey=values_env['storagekey']
localpdfparser=values_env['localpdfparser']
verbose=values_env['verbose']
FILE_PATH = values_env['FILE_PATH']
formrecognizerservice=values_env['formrecognizerservice']
Please create a .env file. In the .env file , put the key in the format = ; example searchservice=
I should have put a .env.example file for clarity. Will put that shortly and explain it.
Added the .env.sample file in the repository.Hope this helps
Here is my little 2 cents', can you do an end-to-end tutorial on how you do this, including creating Azure AI Service, Azure Storage as well as the coding part? 😅😅😅
Great suggestion!
Please. I need it.
I have no idea on Azure or any of its services. I'm just exploring and getting confused
Hi @ambarishg Great video
Will the Azure Search AI store the indexed data or it is stored in StorageAcount
In Azure AI Search
Hi Ambarish, thank you for the very helpful video! I have an enquiry and I hope you could help me with it. So, on the Azure Platform, I have created a a Search Service and an OpenAI Service and connected both in order to be able to ask questions about the data stored on an index. The data is in a Blob Storage container that has multiple folders in it. What I want to do when searching for information, is to filter the folders; I want to choose the folder (either by name or path), so that the search service only returns the content of that folder, and then the openAI service uses that content as a reference to generate an answer to my question. Do you know how I could do that?
In the code we have added metadata while putting data into the search index.
Please add your customized metadata while putting the data into the search index.
While doing the Search on Azure AI , we should provide the filter with the proper metadata and it should filter accordingly. Hope this helps.
May be I should so a followup video explaining this in detail. Thanks again for your questions.
@@ambarishg Thank you so muh for your reply! IT would be perfect if you could show us how to do it. If you can, please show us how to do it on Azure and through code. (I use either Python or C#) Thanks again!
have you tried creating a vector index in ACS? will vector search be better than semantic search?
Good suggestion will try. Usually Hybrid search is better
@ankan54 . Please have a look at the video ruclips.net/video/qJl3IdCKfvE/видео.html where we discuss the following
RAG with Azure AI Search with Azure OpenAI with different search techniques and Langchain with Conversation Chain, Prompt Template and Conversation Buffer
📍 Vector Search
📍 Hybrid Search
📍 Exhaustive KNN Search
📍 Hybrid Search with Semantic reranking
📍 RAG with Azure AI Search with Azure OpenAI
📍 Use Langchain with Conversation Chain
📍 Use Langchain with Prompt Template
📍 Use Langchain with Conversation Buffer
Great video! Very helpful. Instead of azure AI search, if I wanted my retriever to be Qdrant, would it be possible? I have a few files in Qdrant and would like to use Azure open AI to get the output from Qdrant and give the result to the user. If possible, how could I do it?
Yes you can!
ruclips.net/video/-53xYR6yQAY/видео.html This shows RAG with QDRANT [ RAG with Opensource Tools - Qdrant and Mistral in 6 minutes ]. You may please modify this for Azure Open AI
ruclips.net/video/h4F09fWhyhg/видео.html This shows RAG with LlamaIndex - Qdrant and Azure OpenAI in 9 minutes
What if I want the app "also" has the ability to upload our own PDF file and query the content of the just uploaded file through UI?
Yes it is possble
hey can you please put sample env file and explain the parameters
Sure.I will add the ENV File
Added the .env.sample file in the repository.Hope this helps