I think this method returns the answers in one go. How do you stream the response? Also how do you process documents, which is a good feature of Langchain?
You're correct, the method shown in the video returns the answers in one go, primarily because Cloud Functions doesn't support streaming or chunked responses. This is a limitation I'm aware of, and I'm exploring possible solutions. Once I've developed a robust method, I plan to create a follow-up video to share the solution. In the meantime, if you're looking to implement a streaming response, you might consider converting your code to run on Cloud Run, which supports server-side streaming. Regarding document processing, I utilize Cloud Functions to fetch the document initially. After retrieval, I employ langchain.document_loaders to segment the document into manageable parts, embed these parts, and subsequently store them in a vector database. This setup is particularly useful for implementing Retriever-Reader models like RAG (Retrieval-Augmented Generation), allowing for more dynamic and contextually aware responses. Stay tuned for updates, and thank you for your interest.
This is awesome. I'd love to see one where we pass, for example, a Firebase auth user to the function so that the langchain chatflow only utilizes data/vectors/prompts that are specific and limited to the authorized user.
@@cloud_no_9 Nice! I'm playing with the new Firestore Semantic Search extension. It's still very early, but solves the vertexai issue of not storing the vectors with the user's docs. Haven't yet figured out how to a) get a file upload to generate embeddings through this extension or how to integrate it directly with Gemini or other components in the GCP stack, but it's worth pecking around to see if I can make it work. I like keeping the data and the vectors right there under tight auth from Firebase and doing as few external integrations as possible. One nice thing is that each vector embedding with that extension is an item in a list/array of floats/doubles rather than a separate row or document like in a Pinecone or Supabase implementation. Means I don't need a subcollection to manage the vectors under a large file....if I can figure out how to get the extension to work with files (there is a workaround) the way it does with the immediate document content, where it generates a "embedding" list field and a separate tracking collection to do record management, so that changes to data don't get out of sync with vectors.
I think this method returns the answers in one go. How do you stream the response? Also how do you process documents, which is a good feature of Langchain?
You're correct, the method shown in the video returns the answers in one go, primarily because Cloud Functions doesn't support streaming or chunked responses. This is a limitation I'm aware of, and I'm exploring possible solutions. Once I've developed a robust method, I plan to create a follow-up video to share the solution. In the meantime, if you're looking to implement a streaming response, you might consider converting your code to run on Cloud Run, which supports server-side streaming.
Regarding document processing, I utilize Cloud Functions to fetch the document initially. After retrieval, I employ langchain.document_loaders to segment the document into manageable parts, embed these parts, and subsequently store them in a vector database. This setup is particularly useful for implementing Retriever-Reader models like RAG (Retrieval-Augmented Generation), allowing for more dynamic and contextually aware responses. Stay tuned for updates, and thank you for your interest.
This is awesome. I'd love to see one where we pass, for example, a Firebase auth user to the function so that the langchain chatflow only utilizes data/vectors/prompts that are specific and limited to the authorized user.
Yes, that is the main idea. I'm using it with RAG so that each user has access only to documents based on their authorisation level.
@@cloud_no_9 Nice! I'm playing with the new Firestore Semantic Search extension. It's still very early, but solves the vertexai issue of not storing the vectors with the user's docs. Haven't yet figured out how to a) get a file upload to generate embeddings through this extension or how to integrate it directly with Gemini or other components in the GCP stack, but it's worth pecking around to see if I can make it work. I like keeping the data and the vectors right there under tight auth from Firebase and doing as few external integrations as possible. One nice thing is that each vector embedding with that extension is an item in a list/array of floats/doubles rather than a separate row or document like in a Pinecone or Supabase implementation. Means I don't need a subcollection to manage the vectors under a large file....if I can figure out how to get the extension to work with files (there is a workaround) the way it does with the immediate document content, where it generates a "embedding" list field and a separate tracking collection to do record management, so that changes to data don't get out of sync with vectors.
Great vid. Thanks so much. It's really helpful
Thank you, it's the first video I've come across which really summarizes how to properly deploy Langchain to GCP!
Great to hear!
Great explainer!