@krishnaik06: Thanks for the valuable resources. Couple of changes as I was running the code: 1) the model davinici-002 is deprecated. Instead OpenAI advises to use 'gpt-3.5-turbo-instruct'. 2) Pinecone initialization is also changed. For anyone following and coding from here, it was two breaking changes for me.
@@rishabhranjan7687 the code in the video works with pinecode-client==2.2.1 But if you have 4.1.1, just search for langchain pinecode documentation and follow it step by step, it will resolve the issue.
First like and Comment So Grateful for your guidance and contribution to the open-source community at the very need of Hour @KrishNaik Sir 🙏🏻❤️ Love from Bangalore ✨
def chunk_data(docs,chunk_size=800,chunk_overlap=50): text_splitter=RecursiveCharacterTextSplitter(chunk_size=chunk_size,chunk_overlap=chunk_overlap) doc=text_splitter.split_documents(docs) return docs In this part you are not using chunked_data ..need to return - return doc...Is it right?
@krishnaik06, can we do the same with csv's having some kind of predictions from another ML model? Also, can we do some kind of weather report generation from IMD reports - I did try this, but it was not responding accurately. How to ensure the precision?
This is outstanding please share me some videos related to this Build one Q&A bot using open Al and Vector DB provide a couple of docs which should be stored in a vector db and when the questions asked to LLM model the LLM model should retrieve the answer from the vectors stored
Unfortunately 90% of the developer videos are useless because they only build the backend. This is useless to show your manager or a client. Videos that show how to connect a backend to the users front end would be 1,000 times more helpful
Bhai hr cheez thali me paros ke nhi di jaati. Fr kaahe ke developer. Thoda research kro sb mil jaayega. Is information age me bhi specific information ke liye ro rhe h hm to bantadhar h.
Hey Krish, great video! Do you know if it's possible to automatically create a pinecone db index from code? So that you don't have to create them manually
Sir i dont have a credit card. Is it possible to create a project without using the openai api and just by open sourced models from huggingface and langchain. I am quite new in this llm field.
Thanks Krish for the wonderful contributions. Can you please suggest if we could use tabular data with vector db. If any sample code available, can you please share
I did not understand one thing. Does the Vector DB talk to the LLM? We store data from PDF in Vector DB to make it searchable fast. But where is the part where we query Vector DB and send to LLM I could not see any LLM call may be I missed?
Can anyone explain what is the difference between FAISS and Pinecone? In multiple PDF video he used FAISS but here Pinecone. Is there any specific way to use them?
Please recheck your code 10 times before publishing. There is no function named "similarity_search" how your code runs I don't know. Maybe it's a Supercomputer or something.
Sir, lets say I want to deploy this chatbot trained on one pdf which has 2000 pages, can I remove the docuemnt part and only query from the db so that the load time and the embedding cost can be minimzed in production level chat bot, do let me know if you dint get the question, I will try to explain it in a better way.
I have one query - what if I want to connect my local database to a vector database and from that local database, the user can ask the question from the table !! Is this possible ?? if possible then how ??
@krishnaik06 : Thank for the detailed video. I need your help, I am trying to replicate the same with my list of documents. I am able to create vector DB, but when I am querying, something like, Give me the list of employee for which data is uploaded, I am not getting correct answer. Can you please help, how can we solve such type of scenario. your help is appreciated. Thank you
Hi, just want to tell you thay i created a similar project 8 month ago, everything was working but pinecone has migrate to a new version now "init" is no longer a top-level attribute of the pinecone package. I've made lot of reasearch but i don't find any solution to make it working again did you have a idea ?
i dont want to use openai api kindly make it with googlepalm api krish..i have to submit my college project tomorrow just let me know how can i do embedding with the help of palm api key i dont have openai subscription
pinecone is vector database that stores the embeddings, it doesn't offer embeddings itself. there are many open source embeddings online. you can check the huggingface MTEB leaderboard for embedding models.
I am getting an error in the pinecone initiation process. Considering all the process you said I am getting error as pinecone has not attribute init. Can you please help me on how to proceed furhter
Pinecone has now changed its framework. Some elements like "init," "from documents" etc. are now removed. So, the procedure used in this tutorial is now outdated.
You need to return doc in chunk function but you are returning docs which is same as original documents,so you are not giving the text chunks for embeddings but whole documents.
Getting this error while using pinecone vector db, can someone plz help me to resolve this pinecone.init( api_key="a4e02d-d1db-4d66-aaf9-fec1a868", environment="gcp-starter" ) index_name="langchainvector" AttributeError Traceback (most recent call last) Cell In[33], line 2 1 ## Vector Search DB In Pinecone ----> 2 pinecone.init( 3 api_key="a4e0266d-d1db-4d66-aaf9-fe1be7c1a868", 4 environment="gcp-starter" 5 ) 6 index_name="langchainvector" File e:\Gen AI Project\Langchain_using_vectorDB\vectordbvenv\lib\site-packages\pinecone\deprecation_warnings.py:38, in init(*args, **kwargs) AttributeError: init is no longer a top-level attribute of the pinecone package. Please create an instance of the Pinecone class instead.
You probably should stop using “probably” as a filler word. It never makes sense in the context you’re using it. You aren’t “probably” doing something, you “are” doing something.
Can someone please help me, PyPDFDirectoryLoader is not working, Below is the error, tried my best to resolve not happening Traceback (most recent call last): File "C:\LLMAppPinecone\test.py", line 4, in from langchain_community.document_loaders.pebblo import PebbloSafeLoader File "C:\Users\jchristian2\AppData\Local\anaconda3\Lib\site-packages\langchain_community\document_loaders\__init__.py", line 163, in from langchain_community.document_loaders.pebblo import PebbloSafeLoader File "C:\Users\jchristian2\AppData\Local\anaconda3\Lib\site-packages\langchain_community\document_loaders\pebblo.py", line 5, in import pwd ModuleNotFoundError: No module named 'pwd'
Check out other Langchain Projects below
ruclips.net/video/4O1rs7mrNDo/видео.html&pp=gAQBiAQB
Sir U r amazing.
Sir, if possible then make videos on poison attacks on text data like trigger attacks or backdoor attacks.
Aamzing Krishna Bhai...Kya amazing video banaya hai aap ne..Dil jeet liya.
19:12 you need to return doc from the function not docs ( which is the raw input before chunks )?. Nice video and tutorials, thanks
Yes I also noticed same thing
Exactly
@krishnaik06: Thanks for the valuable resources. Couple of changes as I was running the code: 1) the model davinici-002 is deprecated. Instead OpenAI advises to use 'gpt-3.5-turbo-instruct'. 2) Pinecone initialization is also changed. For anyone following and coding from here, it was two breaking changes for me.
Hey there, I'm stuck at pinecode initialization. Can you please tell me how you proceeded?
@@rishabhranjan7687 hey have you been able to get it done?? if yes plz help .
@@rishabhranjan7687 the code in the video works with pinecode-client==2.2.1
But if you have 4.1.1, just search for langchain pinecode documentation and follow it step by step, it will resolve the issue.
Amaizing Knowledge - Thanks Krish for your guidance !!
26:28. We should give the chunks as an input to the function 'from_documents' but not 'doc'
AttributeError: type object 'Pinecone' has no attribute 'from_documents'
how can i resolve this error
@@RaghavSoni-qv5bd you got the solution ??
@@deepaliparochey2285 yes I got the solution of it
First like and Comment
So Grateful for your guidance and contribution to the open-source community at the very need of Hour @KrishNaik Sir 🙏🏻❤️
Love from Bangalore ✨
As simple as that .... Thanks Krish very, very useful stuff.
Informative video like ever
def chunk_data(docs,chunk_size=800,chunk_overlap=50):
text_splitter=RecursiveCharacterTextSplitter(chunk_size=chunk_size,chunk_overlap=chunk_overlap)
doc=text_splitter.split_documents(docs)
return docs
In this part you are not using chunked_data ..need to return - return doc...Is it right?
yes have you checked len(doc)
@@machinelearning3518 in our dir we have one doc or more than one ?
Thank you for such an amazing video!
Amazing learning video
@krishnaik06, can we do the same with csv's having some kind of predictions from another ML model? Also, can we do some kind of weather report generation from IMD reports - I did try this, but it was not responding accurately. How to ensure the precision?
Thank-you dada ❤
17:50 shouldn't we return "doc" instead of "docs" ?
Yes i guess that is a simple mistake i did :).Gald you found it
I too noticed and thats why the Chunks have not been created and the size remains 58 I guess. I am not sure though.
Amazing Tutorial...
Great video. Can we do the same with images and audio, or can we only do similarity searches for those?
Do we have to create vector db all time? Cannot we store the data permanently?
This is outstanding please share me some videos related to this Build one Q&A bot using open Al and Vector DB provide a couple of docs which should be stored in a vector db and when the questions asked to LLM model the LLM model should retrieve the answer from the vectors stored
Disclaimer: if you dont have pinecone api, you can use chromaDB, Sqlite or multiple db but you have to knowledge about schema and tables.
Hey!! Would you like to collaborate to make a llm app? I've already done this tutorial but I am thinking of making a full fledged app.
Unfortunately 90% of the developer videos are useless because they only build the backend. This is useless to show your manager or a client. Videos that show how to connect a backend to the users front end would be 1,000 times more helpful
Right 👍
You can check code with harry he has made with frontend
Bhai hr cheez thali me paros ke nhi di jaati. Fr kaahe ke developer. Thoda research kro sb mil jaayega. Is information age me bhi specific information ke liye ro rhe h hm to bantadhar h.
@@ARkhan-xw8udcan you plz share the link
Then you build the backend, Krish Naik will try to make the front end videos if you feel backend is easier.
Hey Krish, great video!
Do you know if it's possible to automatically create a pinecone db index from code?
So that you don't have to create them manually
can you please solve this error AttributeError: type object 'Pinecone' has no attribute 'from_documents
same error i'm also getting, it seems like they have removed this feature. Now only server less services can create index
I am stuck on "You exceeded your current quota, please check your plan and billing details". Anyone help !
I am stuck on the same @krish Naik. Please help
Actually, added $5 to open ai and waited for 20 mins and tried and it worked. Just created a new API key for clarity.
If you are using old account for accessing chat gpt ..create new account and key.
5 dollars can be used only till 3 months from the date of opening
Excellent Krish 🙏 🙏 🎉🎉
Sir i dont have a credit card. Is it possible to create a project without using the openai api and just by open sourced models from huggingface and langchain. I am quite new in this llm field.
Use Gemini-Pro's API instead of OpenAI.
Actually inside chunk_data you are returning the wrong variable
Thanks Krish for the wonderful contributions.
Can you please suggest if we could use tabular data with vector db. If any sample code available, can you please share
vectors=embeddings.embed_query("what is indian total budget")
len(vectors)
when i excute this it is showing rate limit error
how i do i slove it
Please consider to build an application based on this LLM model
Hi Krish , Bigg thanks for all your contributions, I just have a doubt How to retrieve all docs that has been injected to elastic db?
I did not understand one thing. Does the Vector DB talk to the LLM? We store data from PDF in Vector DB to make it searchable fast. But where is the part where we query Vector DB and send to LLM I could not see any LLM call may be I missed?
Same question
I think we use llm as open embedding model...
Instead of using LLM we used the Langchain dataframe which is the combination of LLM and NLP
Sir please try to apply dockers with it, because it will be really helpful.
Hey!! Would you like to collaborate to make a llm app? I've already done this tutorial but I am thinking of making a full fledged app.
Can anyone explain what is the difference between FAISS and Pinecone? In multiple PDF video he used FAISS but here Pinecone. Is there any specific way to use them?
Please recheck your code 10 times before publishing. There is no function named "similarity_search" how your code runs I don't know. Maybe it's a Supercomputer or something.
Sir I have a error of NoneType object is not callable in retrieve query vectore.similarity search line how to overcome with it
how can i make an LLM model produce output that can be fomatted in the browser for display , like with html tags in it , is that possibel ?
Sir, lets say I want to deploy this chatbot trained on one pdf which has 2000 pages, can I remove the docuemnt part and only query from the db so that the load time and the embedding cost can be minimzed in production level chat bot, do let me know if you dint get the question, I will try to explain it in a better way.
Hey Krish, Could you please create a video on the setilup of pgvector
in chunk_data function you returned "docs" instead of "doc" isn't it wrong?
Below you can see docs=doc
Quiz app related video can you share the link
@krishnaik06 sir please can i have the link of that video which you are talking about at the end of this video??
Openai apikey which we nned to use free tier or subscription, I'm using pro version is 4 apikey error is unauthorized
I have one query - what if I want to connect my local database to a vector database and from that local database, the user can ask the question from the table !! Is this possible ?? if possible then how ??
hii sir , why the pinecone not showing environment value in my pinecone account
same issue, GCP is not included in free tier now, i can only use aws cloud and in that we are not getting any environment variable.
i also have same issue . Can we still use free tier? How abt environment?
Do I need to have a paid account of Open AI to use the OpenAI Embeddings?
Thank You
Error: init is no longer a top-level attribute of the pinecone package.
If I have many files in the same directory how to upload them ? thank you for your amazing vidoes
At the end of the day 😉
Can you pls provide next video link for UI part
@krishnaik06 : Thank for the detailed video. I need your help, I am trying to replicate the same with my list of documents. I am able to create vector DB, but when I am querying, something like, Give me the list of employee for which data is uploaded, I am not getting correct answer. Can you please help, how can we solve such type of scenario. your help is appreciated. Thank you
which blackboard are you using???
Anynpne know???
If we have multiple PDF documents, how that can be handled here ?
Yes I also have same question
Sir can we do this same for csv files. Please answer this question
What is difference between loader vs parser in langchain?
How its able to extract the text from the PDF? Is it using Tesseract or something else?
The similarity_search function is not defined? Any ideas how to define it cause I dont get how it works without the function being defined
hi krish, when i run this code i am getting error like Pinecone has no attribute from_documents please help me to resolve this issue
How run this directly on visual studio by git hub repository
thanku sir
Hi, just want to tell you thay i created a similar project 8 month ago, everything was working but pinecone has migrate to a new version now "init" is no longer a top-level attribute of the pinecone package. I've made lot of reasearch but i don't find any solution to make it working again did you have a idea ?
Yes needed help on this if anyone has a solution to this
I don't want to use model API key. Is there any way to integrate LLM model which is stored in local machine and query the SQL database?
Use llama2 i have already created a video
I get error in vector generating please help me how can I solve this error
How can we overcome that without using openai key?
use gooogle's like .... GoogleGenerativeAI(model="models/text-bison-001",google_api_key=api_key, temperature=0.2)
It works
Hello Krish,
Kindly request if you can share link to colab file if possible.
hello krish, I was tried this code facing some issue with Pinecone.from_documents. there is no library called from_documents in pinecone.
I am also getting same error
Hi! Guyz when I run the code I got some errors. Please help me
This video is outdated. The pinecone API has changed significantly and this tutorial wont work
Did you find the updated solution?
Awesome bro!
hi sir in gihub source code of test.ipynb not avialabel with full code it has only import libirary
Updated the code
i dont want to use openai api kindly make it with googlepalm api krish..i have to submit my college project tomorrow just let me know how can i do embedding with the help of palm api key
i dont have openai subscription
Is there any open source embeddings we can use instead of Pinecone
pinecone is vector database that stores the embeddings, it doesn't offer embeddings itself. there are many open source embeddings online. you can check the huggingface MTEB leaderboard for embedding models.
I am getting an error in the pinecone initiation process. Considering all the process you said I am getting error as pinecone has not attribute init. Can you please help me on how to proceed furhter
Pinecone has now changed its framework. Some elements like "init," "from documents" etc. are now removed. So, the procedure used in this tutorial is now outdated.
it should be Pinecone instead of pinecode in pinecode.init
@@syedmuzammilahmed6872 So how do we load a document object into pinecone now then? Since Pinecone.from_documents(docs, embeddings) no longer work
@@ggg9gg Anyone found the solution for this ?
Pinecone instance is not being created and how to do in newer version in v3.2.2 please tell
Did you get the solution ?? if yes please help
@@deepaliparochey2285 use older version of langchian and pinecone it will work
The pinecone initialization step described in this video is not working. Any updated video or code please?
did you get the solution ?
from pinecone import Pinecone
pc = Pinecone(api_key=pinecone_api_key)
index = pc.Index(index_name)
Why mine is taking so much time to print the answers
pinecone is modified, this code does not work now, needs to be updated
AttributeError: type object 'Pinecone' has no attribute 'from_documents'
Yeah same issue, something wrong with langchain and pinecone once they have upgraded the version
🎉🎉🎉🎉
Have you anyone give RateLimitError , 429 error?
Yup. Stuck on the same on open AI 429 error.
Actually, added $5 to open ai and waited for 20 mins and tried and it worked. Just created a new API key for clarity.
29:38 😂😂
You need to return doc in chunk function but you are returning docs which is same as original documents,so you are not giving the text chunks for embeddings but whole documents.
Ha bhai its a minor mistake u can update the code :)
Getting this error while using pinecone vector db, can someone plz help me to resolve this
pinecone.init(
api_key="a4e02d-d1db-4d66-aaf9-fec1a868",
environment="gcp-starter"
)
index_name="langchainvector"
AttributeError Traceback (most recent call last)
Cell In[33], line 2
1 ## Vector Search DB In Pinecone
----> 2 pinecone.init(
3 api_key="a4e0266d-d1db-4d66-aaf9-fe1be7c1a868",
4 environment="gcp-starter"
5 )
6 index_name="langchainvector"
File e:\Gen AI Project\Langchain_using_vectorDB\vectordbvenv\lib\site-packages\pinecone\deprecation_warnings.py:38, in init(*args, **kwargs)
AttributeError: init is no longer a top-level attribute of the pinecone package.
Please create an instance of the Pinecone class instead.
Pinecone has removed init() in latest update. please use pinecone-client==2.2.4
👍🙏
Man I'm facing 10000 errors, how could every people speaking out like they done it in 1st attempt
because some functions are depricated and may be you are running as usual code.
Sir how to get openai key sir
Please use free api's
You probably should stop using “probably” as a filler word. It never makes sense in the context you’re using it.
You aren’t “probably” doing something, you “are” doing something.
Can someone please help me, PyPDFDirectoryLoader is not working, Below is the error, tried my best to resolve not happening
Traceback (most recent call last):
File "C:\LLMAppPinecone\test.py", line 4, in
from langchain_community.document_loaders.pebblo import PebbloSafeLoader
File "C:\Users\jchristian2\AppData\Local\anaconda3\Lib\site-packages\langchain_community\document_loaders\__init__.py", line 163, in
from langchain_community.document_loaders.pebblo import PebbloSafeLoader
File "C:\Users\jchristian2\AppData\Local\anaconda3\Lib\site-packages\langchain_community\document_loaders\pebblo.py", line 5, in
import pwd
ModuleNotFoundError: No module named 'pwd'