A practical application leveraging Langchain and BigQuery Vector Search
HTML-код
- Опубликовано: 25 июл 2024
- Today, I'm thrilled to share insights into the integration of Langchain and BigQuery Vector search through a practical application that I've developed. This video presentation goes beyond theoretical discussion, offering a hands-on look at leveraging these cutting-edge technologies.
I cover critical topics like Large Language Models (LLM), Embeddings, and Vector Search, which are fundamental to the application's architecture. You'll see how these components work together to provide highly accurate and context-specific information, a key aspect of delivering real value in any business setting.
Furthermore, I discuss the broader implications of adopting this technology pattern within the industry, highlighting straightforward opportunities, areas requiring careful consideration for success, and high-risk challenges that necessitate diligent attention to ensure optimal solution performance.
I hope you find the content of this video useful :)
* 01:20 - What is Langchain
* 02:17 - Technical terms used in this presentation
* 03:51 - Use case explained
* 06:35 - Architecture
* 12:08 - Demo
* 18:53 - Code walkthrough
* 25:33 - Code walkthrough. Schema of the table on BigQUery
* 32:49 - Opportunities
* 40:01 - Summary
Code base: github.com/rocketechgroup/lan...
Slides: docs.google.com/presentation/... 
Наука
Wow, this video is incredibly insightful and informative! 👏 I've learned so much and am grateful that you've shared this valuable content with us.
Just a quick question: Could I apply these concepts to create a conversation app that details the findings in the search results? Looking forward to your guidance on this.
Glad you found it useful!
I don’t see why not, but as I mentioned, for a conversational app, I assume you are talking about an app that is consumer (real customers) facing, the concept is exactly the same but you need to change the vector db to something that supports highly concurrent workload. So BigQuery is out of the picture, you can look at vertex AI vector search and also AlloyDB which I am hearing a lot lately. I haven’t tried either yet but as far as I know they are both valid approach for consumer apps supports highly concurrent workload. The docs for alloyDB is here cloud.google.com/alloydb/docs/ai/work-with-embeddings
Thank you for your valuable insights and guidance!
You are welcome ;)
Hi Richard, tysm for the video, it was really helpfull! Im a jr data scientist working on a customer service chatbot using dialogflow cx and some webhooks for the RAG (product recomendations, stock, price...). My original vector store was VertexAI Vector Search, since its really expensive im looking for other options like BQ Vector Search but you mentioned that a consumer traffic is an issue. Do you have any vector store that you can recomend? Im trying to mantain my solution inside my GCP. Thank you again 🙌
Sorry just realised I forgot to reply. I was actually going to look at vertex AI vector search next, as it is one of the good options for vector search, can you provide some metrics on what would you consider as expensive?
There are other options like Postgres, elastic search, you can see what you can use from the list langchain has integration with which covers most of them. python.langchain.com/docs/integrations/vectorstores. If you want to control cost better, you can consider the services aren’t based on volume but cost of storage engine. Although you have trade offs on managing infrastructure and also optimising for performance. So it’s more of try and error not a straight answer
@@practicalgcp2780 id like to hear more about vertex vector search since i cant find good deep dives on it :(
@@practicalgcp2780 The endpoint machine I'm using is quite large (16 vCPUs, 64 GiB). While I'm only working with half the intended volume of products at the moment, the cost is still around $1k per month. I suspect there might be a default parameter set during creation that prevents me from selecting a smaller machine type.
I read the integrations and I think my next step is AlloyDB, the good thing about Vertex AI Vector Search is that is really fast so I need to see the difference between those two. Im looking forward to your videos since I haven't seen much content about Vertex AI Vector Search. Thank you for the reply 🙌
May I ask one more question regarding this matter?
I would like to implement permissions for a chat application concerning access to documents.
For example, a person should only have access to certain tables or specific fields within those tables, and if they don't have permissions, they wouldn't be able to search. Do you have any suggestions or keywords that might help with this issue?
Thank you very much for your assistance
That is something you have to do through some sort of RBAC implementation (role based access control). That isn’t anything to do with the search, it’s more on mapping out the role of a user through logging in like what most applications do today. Then depends on the role, you can add specific filters in the search queries, like filtering via certain metadata or have a set of tables you can restrict based on roles etc.
Sure, I understand that. However, I'm looking for a service that can assist me with implementing RBAC.
ok I see. I think it really depends on what you are using. For example, if you are building a backend with Python. You can use Django which has a RBAC module, but generally any framework would have some sort of RBAC component you can use. If it’s an internal app (like for within the company use) then you can simplify things by just using IAP, but IAP isn’t suitable for external consumer applications
@@practicalgcp2780 thank you for your answer!
😎 "Promosm"