How To Integrate OpenAI With Azure Vector Search aka Azure Cognitive Search

Check out this solution if you want to get started with OpenAI integration with Azure vector search database. This video covers end to end solution starting from generating Azure instances to getting response from OpenAI API.
Blog: www.shwetalodha.in/
Medium: / shweta-lodha
* REFERRAL LINK ************
Medium referral link: / membership
* REFERRAL LINK ************
###### MORE PLAYLISTS ######
⭐Python for beginners: • #1 Python for Beginner...
⭐Python Pandas: • #1 Python Pandas: Intr...
⭐Python tips and tricks: • Python Tip: Take Multi...
⭐Jupyter tips & tricks: • Jupyter Tip: Run Termi...
⭐Microsoft Azure: • Know Response Time Of ...
⭐Azure ML and AI: • Getting Started with I...
⭐Visual Studio Code a.k.a. VS Code: • How to get started wit...
#openai #chatbot #azure

[@shweta-lodha]

Code snippets and bit of explanation shweta-lodha.medium.com/step-by-step-tutorial-to-integrate-openai-with-azure-cognitive-search-vector-search-4d75b400675c

[@jorgeanicama8625]

Very straight to the point. Just the video i needed to connect many isolated concepts! Thank you very much. !

[@shweta-lodha]

Glad you find it useful. Stay tuned for more such bites 👍

[@soubarnobanerjee8257]

Just loving your contents! There are a lot of grey areas onto the LLMs and how they work if we integrate Azure services, especially the vector DB part. Using FAISS, Chroma, etc are good for POCs, that too performed internally.... but something which gets accepted in enterprise level is the one which you are talking about in this video!!
Thank you

[@shweta-lodha]

Thank you 😊

[@MrRugev]

Thank u!!!!, your videos have helped me a lot.

[@shweta-lodha]

I'm so glad!

[@user-ru4nb8tk6f]

This is so good, thank you so much

[@shweta-lodha]

Glad to know that 🥰

[@joannezhu101]

Thank you Shweta, really helpful for me to understand how the Cognitive search works. I have another question, wouldn't it be easier to use Azure openAI instead of OpenAI as the chat language model towards the end? I'll duplicate the code on my end. Thank you so much for sharing this with us.

[@user-cs2wy2wr1r]

Hi Thank you for the video ,i am getting httpresponseerror invalid parameter can u help me with this?

[@user-tf1iq7mr1w]

thanks for the video.

[@shweta-lodha]

You're welcome

[@user-xz1jh9qv1k]

thanks this is really good where is generate_embeddings function located and which embedding model are you using? share the notebook?

[@deepikahari3790]

I could not find "generate_embeddings" function definition.

[@joellim1]

Hi Shweta, your content is very clear and useful! How might we get the full .ipynb source code so we can try out the solution ourselves? For instance, generate_embeddings function is not shown.Thank you for all that you do!

[@shweta-lodha]

It is there on my Medium blog. Medium link in present on my profile page

[@joellim1]

Many thanks@@shweta-lodha

[@permanazainal6003]

@@shweta-lodha medium member only

[@user-fd3ie2qk7x]

Hi @shweta lodha, can we rename the index name in azure cognitive search.

[@shweta-lodha]

You can’t

[@dheerajsachdeva5735]

Hi! Can you please share the notebook that you used in the video? The medium site is for members only.

[@shweta-lodha]

That is the only option I have today. Sorry for that

[@Haraker21]

Hi, how would you suggest to implement this solution with multiple documents?

[@shweta-lodha]

It doesn’t actually matter from DB side, whether you are taking single or multiple docs. It only differs at a point where you are reading your docs. If you have multiple docs, you can load using some directory loader

[@abhayrajput9546]

What is generate_embeddings ?, not there in your blog

[@shweta-lodha]

This implementation varies based on your choice - whether you are going with OpenAI embedding, hugging face or any other. Purpose of this function is to generate the embedding for the given text

[@hrishabhgupta7568]

I am following as per the tutorial but getting this error while creating the Index:
vector_search_configuration is not a known attribute of class and will be ignored

[@shweta-lodha]

Are you sure, it’s an error and not a warning?

[@pavan1006]

In production, how do you deploy this python code ?

[@shweta-lodha]

You can create a web app and do it

[@HariTummuri]

can we use pdf files in the sameway?

[@shweta-lodha]

No, reader would be different

[@user-ru4nb8tk6f]

This line is not fully displayed in the video could you kindly provide the rest of the line, please?
SearchField(name="embedding", type=SearchFieldDataType.Collection (SearchFieldDataType.Single), searchable=True, vector_search)
Thanks

[@user-ru4nb8tk6f]

as well as generate_embedding function

[@shweta-lodha]

I don't have the complete code, but I hope this snippet would work:
fields = [
SimpleField(name="documentId", type=SearchFieldDataType.String, filterable=True, sortable=True, key=True),
SearchableField(name="content", type=SearchFieldDataType.String),
SearchField(name="embedding", type=SearchFieldDataType.Collection(SearchFieldDataType.Single), searchable=True, vector_search_dimensions = 1536, vector_search_configuration ="my-vector-config")
]
index = SearchIndex(
name=index_name,
fields=fields,
vector_search=vector_search
)

[@shweta-lodha]

response = openai.Embedding.create(
input="text", engine="text-embedding-ada-002")
embeddings = response['data'][0]['embedding']

[@user-ru4nb8tk6f]

Thanks!@@shweta-lodha