This did not work for me. When I initiate the Upsert API call to Pinecone it gives me a 400 error when I change the dimensions to a dynamic field. Also, "Vector" does not show up as a data field in the Workflow. Only "Values".
The vector needs to contain 1536 vectors even in the initialization. The one they have in the cURL command actually works but copy and paste it manually, don't use that one-click copy. I'm actually doing a very similar step-by-step tutorial on my channel here very soon. Feel free to check out!
When using a local vector DB ( pinecone ) you are stil using OpenAI as processing for the incoming text. Are there any concerns about privacy loss for the data in your own pinecone database? Meaning - can you use pinecone for internal data and not worry about openAI or others having some kind of inferred access ?
Great question! I'm not really sure "what happens" on the OpenAI Embeddings level (or any other model) tbh...but Pinecone does need embeddings in order to store and process data (it's a vector DB), so you have to feed it vectors. I'd check their docs for a proper answer!
Great thanks! How would you set things so that the LLM decides that an action is needed from what it is being prompted? Would you fine tune it to assess from different texts if a Google Meeting (example) is needed? Then return “set up meeting for mm/dd/yy 3:00pm” and have Bubble interpret this to send an API call to Google to create this mtg?
Great question! To clarify, this would allow to use natural language to achieve things like booking a meeting - correct? At the top of my head you might need to train your model to "know" what you mean by "meeting" etc. so that it can be prompted. Google API should be able to ingest that metadata. Very interesting idea indeed!
you mean the result is exactly the text you upserted? if so, it's bc you're not passing the query results back into an LLM (OpenAI Chat/Completion) therefore there's no interpretation but simply the data you provided
Great video! Do you have a recommendation for how to format and upsert a more complex data source? The data set I'd like to use is in Google Sheets at the moment. Trying to figure out if I convert it to PDF, DOC, or just copy/paste values as text similar to the way you've done in your example.
This is an awesome video. Thank you! When I go to add the state "value" in the workflow, I don't see the "Result of step 2 matches: each item's metadata" as an option. I only have ID, Value, Score, and Raw Text. Am I missing something during the setup to bring this option forward?
So this doesn't use GPT, we're just querying what's in Pinecone based on the Vector returned from the question? And how does it determine what to return?
This uses OpenAI Embeddings to turn text into vectors, since Pinecone or other vector databases only understand vectors. This is how "similarity" is determined when querying results. You could pass that query to an LLM like OpenAI Chat/Completions for interpretation (rephrasing)
Can you provide a step-by-step document for this...I'm a beginner & found a little tough to follow this? It would be helpful if a document explaining the flow & use of each s/w could be provided. Thank you in advance!
Can you show us how to upload files via the pinecone ui. Formatting the data etc. Also I know your a babble man but if you did a video on incorporating this into Flutter Flow your views would be crazy!! FF is on fire lately.
Hey Chris, haven't used the UI (only API) but I'm sure it's as easy as loading the file - that being said, you need to create embeddings for your files first, for which you will need to use something like OpenAI's Embeddings. Also haven't had a chance to play with FF, but if it supports cURL API calls, it'll be identical to this process, at least on the API side. Thank you for watching!
Do you think it's bc of the token limitations or the model itself? I haven't looked into HF with Bubble yet, but I hear about it quite a bit in other environments!
Dear Friend, subed, liked, commented! Rare gem. could you have another video that show how to deal with input longer than maximum content length? 8191 tokens, in bubble. Thanks alot
![Noob To Max With DRAGON REWORK In Blox Fruits [FULL MOVIE]](http://i.ytimg.com/vi/LnBMOinoOvA/mqdefault.jpg)
Wow this is a great video and needs more than 5 minutes because you’re cooking with oil here; I like it
Informative. Would love a more informative deep-dive on this.
Thank you! Next video coming up! It’ll be a lot more in depth
Definitely! Yours is the only video I could find on this. My project is stalled until I can get this to work.😢
This did not work for me. When I initiate the Upsert API call to Pinecone it gives me a 400 error when I change the dimensions to a dynamic field. Also, "Vector" does not show up as a data field in the Workflow. Only "Values".
Seconded on this. Did you find a solution?
The vector needs to contain 1536 vectors even in the initialization. The one they have in the cURL command actually works but copy and paste it manually, don't use that one-click copy. I'm actually doing a very similar step-by-step tutorial on my channel here very soon. Feel free to check out!
@@cerumai Yes, I ended up doing that and got it to work. thx.
indeed 1536 dimensions is what this particular model (OpenAI) mandates
Did you solve this? the comment below addresses what I think might be the issue
So beautiful 😍😍 exactly what I was needing!!!!
When using a local vector DB ( pinecone ) you are stil using OpenAI as processing for the incoming text. Are there any concerns about privacy loss for the data in your own pinecone database? Meaning - can you use pinecone for internal data and not worry about openAI or others having some kind of inferred access ?
Great question! I'm not really sure "what happens" on the OpenAI Embeddings level (or any other model) tbh...but Pinecone does need embeddings in order to store and process data (it's a vector DB), so you have to feed it vectors. I'd check their docs for a proper answer!
Great thanks! How would you set things so that the LLM decides that an action is needed from what it is being prompted? Would you fine tune it to assess from different texts if a Google Meeting (example) is needed? Then return “set up meeting for mm/dd/yy 3:00pm” and have Bubble interpret this to send an API call to Google to create this mtg?
Great question! To clarify, this would allow to use natural language to achieve things like booking a meeting - correct? At the top of my head you might need to train your model to "know" what you mean by "meeting" etc. so that it can be prompted. Google API should be able to ingest that metadata. Very interesting idea indeed!
@@damianoredemagni7701 Right. Prompting the LLM to think through an input and assess if an action is needed as its response.
Very helpful video. The Query response produces the Upsert info. What could be the hiccup if I’m replicating the steps you provided?
you mean the result is exactly the text you upserted? if so, it's bc you're not passing the query results back into an LLM (OpenAI Chat/Completion) therefore there's no interpretation but simply the data you provided
@@damianoredemagni7701 That worked. Thank you!
Great video! Do you have a recommendation for how to format and upsert a more complex data source? The data set I'd like to use is in Google Sheets at the moment. Trying to figure out if I convert it to PDF, DOC, or just copy/paste values as text similar to the way you've done in your example.
Check out fine-tuner.ai - should cover all those use cases
This is an awesome video. Thank you! When I go to add the state "value" in the workflow, I don't see the "Result of step 2 matches: each item's metadata" as an option. I only have ID, Value, Score, and Raw Text. Am I missing something during the setup to bring this option forward?
did you check in the API connector that the field is not "Private"?
Same here. Any fix?
You need to initialize the call for the "matches" option to appear. Do this using the initial vector values of cURL, before replacing with .
So this doesn't use GPT, we're just querying what's in Pinecone based on the Vector returned from the question? And how does it determine what to return?
This uses OpenAI Embeddings to turn text into vectors, since Pinecone or other vector databases only understand vectors. This is how "similarity" is determined when querying results. You could pass that query to an LLM like OpenAI Chat/Completions for interpretation (rephrasing)
Are we doing a fallow up soon?
I should be able to make something soon, currently doing some LLM research - stay tuned!
Can you provide a step-by-step document for this...I'm a beginner & found a little tough to follow this? It would be helpful if a document explaining the flow & use of each s/w could be provided. Thank you in advance!
Good idea! A blog post would be a good alternative!
@@damianoredemagni7701 - Love the video - did you ever publish the detailed instruction blob?
@@gahlberkooz9922 thank you! I have not, yet :)
I want to use openai generation to process the chunks after retrieving them, how can I do this?
You'd have to add an additional Chat API call and give it a relevant prompt. This is a neat technique to get dynamic, but relevant, responses.
Amazing!!
How could you train the model with pdfs or urls... or you just have to copy de info in that box. ? Thanks
Trying a few things to make it swift! Hopefully soon! (Everyone asking for PDF)
@@damianoredemagni7701 ok, thanks!
@@jguillengarcia Seconded. Would love to see this with PDFs!
Can you show us how to upload files via the pinecone ui. Formatting the data etc.
Also I know your a babble man but if you did a video on incorporating this into Flutter Flow your views would be crazy!! FF is on fire lately.
Hey Chris, haven't used the UI (only API) but I'm sure it's as easy as loading the file - that being said, you need to create embeddings for your files first, for which you will need to use something like OpenAI's Embeddings. Also haven't had a chance to play with FF, but if it supports cURL API calls, it'll be identical to this process, at least on the API side. Thank you for watching!
🎉🎉🎉🎉🎉🎉
Can you create a tutorial where it also fetches the documents along with the answer?
for document upload/retrieve with Bubble take a look at fine-tuner.ai
This works great, but training this thing from scratch is impossible. A great follow up video to this would be adding embeddings from Hugging Face.
Do you think it's bc of the token limitations or the model itself? I haven't looked into HF with Bubble yet, but I hear about it quite a bit in other environments!
Dear Friend, subed, liked, commented! Rare gem. could you have another video that show how to deal with input longer than maximum content length? 8191 tokens, in bubble. Thanks alot
Appreciate it! Video is actually in the works! Stay tuned!
Now what?
Hey Damiano, I would love to get in touch with you about a no code partnership. Do you have a email or website where we can discuss ?