My sincere thanks to you for making such an insightful video, sir It would be great if you could make a video on how to create a custom LLM model. Currently, we are using another company's LLM, so I want to create a new one and publish it to the public or Huggingface. love from Gujarat
For Big Database again inform queries by Prompt(like train LLM again) is huge job . May other way like Fine Tuning Model on Specific domain or Retrain it.
31:53 Facing error at vectorstore = Chroma.from_texts(to_vectorize, embeddings, metadatas=few_shots) ValueError: Expected EmbeddingFunction.__call__ to have the following signature: odict_keys(['self', 'input']), got odict_keys(['self', 'args', 'kwargs'])
can we ask context based response? eg : Q1 . "how many XS size thsirt are there?" ans 50, Q2. "and S size?" LLM will understand the context of Q2 which is 'SiZE'
I want to make a report generating software using LLM on my ERP where nearly 200 tables are exists. In my report generating software I give a propmt and LLM will analysis this promps and make a sql query on the basis of my ERP databases. Can I do it using this procedure?
please check a playlist link in the video description (I have 3 end to end projects already on LLM in education sector, finance and this one is on retail)
I'm able to run it on jupyter notebook but pycharm throws this error for the wrapper: validation error for SQLDatabaseChain database instance of SQLDatabase expected (type=type_error.arbitrary_type; expected_arbitrary_type=SQLDatabase)
I'm facing an issue with sentence-transformer. Even after installation when it comes to HuggingFaceEmbeddings, it is throwing error install no module named torch._c and also says to install sentence-transformers. Python, torch versions are matching though.
Is this free slot prompting scalable in production environments? There can be so many tables, do many different kinds of possible queries. We can not keep asking queries to prompts. Also these few shots examples would need to be part of the LLM context will be costly.
Another superb video. Can we build "End to End LLM Project Using Langchain, OpenAI in Finance Domain" which you shown 1 month ago with Google PaLM? I am asking because as you said google PaLM is free. So those who doesn't have OPENAI paid account then they can practice it. If possible then please make a video on it. Thanks in advance
Hey yes for that finance project you can use Google palm instead of open ai. you can get it done with few lines of code change. I am not sure if I will have that to make a separate video on that. consider this as an exercise for yourself 😃
@@codebasics Thank You very much for your kind support. You are a real gem of this industry. You are putting your great effort for India's bright future.
sir how to give table and column name description to llm since most of the time the names are assigned based on domain knowledge so how to give custom description
Hi, how I will decide that which GPT version will suitable for my project as below code? Please send any guideline/rule regarding this. Thank you. chatllm=ChatOpenAI(openai_api_key=os.environ["OPEN_API_KEY"],temperature=0.6,model='gpt-3.5-turbo')
@ 14:33 while executing this db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True) i am facing an error ValidationError: 1 validation error for SQLDatabaseChain database instance of SQLDatabase expected (type=type_error.arbitrary_type; expected_arbitrary_type=SQLDatabase) please someone help me
OSError: [WinError 126] The specified module could not be found. Error loading "C:\Users\Moin\anaconda3\Lib\site-packages\torch\lib\fbgemm.dll" or one of its dependencies. i m getting this error, not able to tackle
what if the quantity of t-shirt decreases which means someone buys after some time passed, do we need to change few_shots list's values and rerun the whole process? or llm figure out by its own?
Facing error at vectorstore = Chroma.from_texts(to_vectorize, embeddings, metadatas=few_shots) ValueError: Expected EmbeddingFunction._call_ to have the following signature: odict_keys(['self', 'input']), got odict_keys(['self', 'args', 'kwargs'])
Re-check this things: 1. from langchain.vectorstores import Chroma this is correct 2. formate of your few_shots dict it correct. 3. to_vectorize = [" ".join(example.values()) for example in few_shots] This is correct. Might solves the issue
In production answers might change every day, it will be not possible to change answers every day right? And what about if I need data in table format, It will only answer in one single word, if we need data of 2 or more columns, so what would be the approach for it?
Error of this line :--to_vectorize = [" ".join(example.values()) for example in few_shots] This is the error :---------------------------------------------------------------------------- TypeError Traceback (most recent call last) Cell In[31], line 1 ----> 1 to_vectorize = [" ".join(example.values()) for example in few_shots] Cell In[31], line 1 ----> 1 to_vectorize = [" ".join(example.values()) for example in few_shots] TypeError: sequence item 3: expected str instance, dict found If I change above line with this to_vectorize = [{" ".join(str(value) for value in example.values() if isinstance(value, str))} for example in few_shots] this is working fine but in next line vectorstore = Chroma.from_texts(to_vectorize, embeddings, metadatas=few_shots) it is giving me this error 86 texts = list(map(lambda x: x.replace(" ", " "), texts)) 87 if self.multi_process: 88 pool = self.client.start_multi_process_pool() AttributeError: 'set' object has no attribute 'replace'
I resolved it using the following changes:- in few_shots, make sure to take only the results from the qns1, qns2, qns3, qns4, qns5 . i;e., qns4['result'] in-place of qns4 and so on. and same for others.( if qns is alread a string value and not dictionary, then leave it.)
Hi , this video was super helpful for my work but can we retieve the SQL query without running it, actually I want only the SQL Query. I tried with logger but failed. Any good idea would help a lot.
I have no science background in my school level and i completed my post graduation.is it eassy to shift data science filed.. If i can learn data science certificate programm.. If data science short term course are available in data science filed
I am trying to join the list but i am getting error like this "sequence item 3: expected str instance, dict found" how can i solve this error please help me to comeout from this
@@codebasics as the result in the browser I get this output "Question: what is the quantity of red tshirt? SQLQuery: SELECT sum(stock_quantity) as total_quantity FROM t_shirts WHERE color = 'Red'" not get output like you
Sir you are using a GPT api key internally how they work and what is providing in the code when you use the chain function then you are not going to encode query. And you pass LLM = LLM means what??
![Tee Grizzley - Blow for Blow (feat. J. Cole) [Official Video]](http://i.ytimg.com/vi/wNLN5xsx5dc/mqdefault.jpg)
![Business Analyst Full Course [2024] | Business Analyst Tutorial For Beginners | Edureka](/img/1.gif)
Check out our premium machine learning course with 2 Industry projects: codebasics.io/courses/machine-learning-for-data-science-beginners-to-advanced
Bro first line of code hi run nhi ho rha h pleade help kr do mera final year project h mera 😢😢😢
@@ritikkumarsah7 did you figure out how to do this bro
@@SatwikVarmaKunaparaju Yes. Some say API service is closed and some say you have to pay
This is one of the best tutorial I watched so far, it is comprehensive with perfect example. feeling lucky after going though this.
Thank you sir, I took this project as base for my major project in college and it got selected , you helped me in my majors thanks sir thanks alot.
You really are a fantastic teacher! Thank you for the time and effort that you have put in to create this outstanding video.
Its such an amazing video. You explained such a complex topic in a simple language and the demo alongside nailed it. !! Hats off to you!
Excellent !!! Can you make a video on (1) How to deploy custom LLM applications to Production (2) How to Evaluate LLM models
My sincere thanks to you for making such an insightful video, sir
It would be great if you could make a video on how to create a custom LLM model. Currently, we are using another company's LLM, so I want to create a new one and publish it to the public or Huggingface.
love from Gujarat
Please make a video from database to deployment...❤
i am getting error .. while importing file from .SQL (mySQL ) to MS SQL SERVER
U always explain very easily😊
Thanks you Sir. you make really best videos with clear concepts. please make more projects it will really help
Hi Dhaval Please make a video on How to Evaluate LLM models? Which metrics need to use?
can I use custom databases which can be uploaded by the user?
For Big Database again inform queries by Prompt(like train LLM again) is huge job . May other way like Fine Tuning Model on Specific domain or Retrain it.
31:53
Facing error at
vectorstore = Chroma.from_texts(to_vectorize, embeddings, metadatas=few_shots)
ValueError: Expected EmbeddingFunction.__call__ to have the following signature: odict_keys(['self', 'input']), got odict_keys(['self', 'args', 'kwargs'])
pip install chromadb==0.4.15
@@rohaanmanzoor3268even after installation am also facing the same error
still getting same error. did u fix it?
@@nandanab525 hey ! did you fix it? I'm stuck here.
heres what I did to resolve
!pip install -U langchain chromadb langchain-community
then restarted kernel and all set
This is goldmine of information!!
What if we change database every day like we insert or delete? This will work?
@@loading757Why will you change DB everyday?
@@2dapoint424 i mean if we alter, insert delete or make changes in db?
@@2dapoint424 😂😂😂😂
Bro first line of code hi run nhi ho rha h pleade help kr do mera final year project h mera 😢😢😢
This seems like a really good video, I'll be watching this later today. Wondering if you have a project where Databricks is used as data source?
hats off , took me a while to understand fewshotprompttemplate
How can I do it if I have elastic search instead of mysql database?
Thanks for the video. Concepts and flow explaination is really nice. Its taking some time to digest all of this.
Dear sir, thank you very much for great tutorial! Keep it going on!
can we ask context based response? eg : Q1 . "how many XS size thsirt are there?" ans 50, Q2. "and S size?" LLM will understand the context of Q2 which is 'SiZE'
Very nice use case! Quick question though, we don't we create a dashboard out of this DBMS?
Will this update the current data or we only deal with the historic data??
I want to make a report generating software using LLM on my ERP where nearly 200 tables are exists. In my report generating software I give a propmt and LLM will analysis this promps and make a sql query on the basis of my ERP databases. Can I do it using this procedure?
GREAT project!!!!!!!! Waiting for more LangChain Projects
please check a playlist link in the video description (I have 3 end to end projects already on LLM in education sector, finance and this one is on retail)
Any one have updated code
How humble dhaval sir who keeps us motivated
Thank you, Happy Learning.
@@codebasicsbut sir if we need answers of multiple coloumn what will we do? And database changes everyday, is it incorporate with our model?
I am not able to run, please help me, this is my final year project 😢😢😢 If anyone can help me, I am ready for paying.
@@ritikkumarsah7 ping me in hindi saat nau saat saat paanch saat ek do aat che
I'm able to run it on jupyter notebook but pycharm throws this error for the wrapper:
validation error for SQLDatabaseChain
database
instance of SQLDatabase expected (type=type_error.arbitrary_type; expected_arbitrary_type=SQLDatabase)
i got this error too, if you find a solution can you tell me how you managed? thank you in advance
Could you resolve the error?
@shreyasreedhar1225
@user-du4kl4by3y
@vms7597
Could you resolve the error?
I am not able to run, please help me, this is my final year project 😢😢😢 If anyone can help me, I am ready for paying.
module 'google.generativeai' has no attribute 'generate_text'
please how can i fix it?????????
Excellent video Dhaval Sir. 👌👏🏻👏🏻
I'm facing an issue with sentence-transformer. Even after installation when it comes to HuggingFaceEmbeddings, it is throwing error install no module named torch._c and also says to install sentence-transformers. Python, torch versions are matching though.
Hey bro run hua kya.. ye mera final year project hai. Run ho rha hai help kr do please i am ready for paying 😢😢😢
Can I perform the same with MongoDB (No SQL) too? Does langchain support that @codebasics?
which google-generativeai version have you got installed? i am getting tons of errors due to version mismatch...
Same here bro
I am not able to run, please help me, this is my final year project 😢😢😢 If anyone can help me, I am ready for paying.
I am getting issue with langchain libraries , pls advice python version and import lib versions -- Thanks
Dont trust this guy. You will stuck on midway 😢😢
Is it possible to use hugging face llm model api key too
Is this free slot prompting scalable in production environments? There can be so many tables, do many different kinds of possible queries. We can not keep asking queries to prompts. Also these few shots examples would need to be part of the LLM context will be costly.
Superb
Can you please create a video using llama3 for getting answers from a pdf document
Can we use HuggingFace InferenceClient?
Obviously
Nice teaching ❤
Well explained
Facing issue in establishing a connection b/w python and SQL. Can someone help?
hi sir, while doing langchain i found that model is passing unwanted quotes to the db. Can you suggest any solution to rectify this problem?
Another superb video. Can we build "End to End LLM Project Using Langchain, OpenAI in Finance Domain" which you shown 1 month ago with Google PaLM?
I am asking because as you said google PaLM is free. So those who doesn't have OPENAI paid account then they can practice it.
If possible then please make a video on it.
Thanks in advance
Hey yes for that finance project you can use Google palm instead of open ai. you can get it done with few lines of code change. I am not sure if I will have that to make a separate video on that. consider this as an exercise for yourself 😃
@@codebasics Thank You very much for your kind support. You are a real gem of this industry. You are putting your great effort for India's bright future.
Did you figured it how to use Google PaLM instead of OpenAI
I have been working on it for a long time but getting error at embedding
@@codebasicssir can this be connected sql server db?? .. I am using quite long
Can we use langchain in corporates. I mean using API access key may be not secure to use in corporate industries because of data security
how to print associated SQL query for the question in streamlit?
I have a doubt. Will this send my Database contents/schema along with API request to LLMs for queries I ask ?
sir how to give table and column name description to llm since most of the time the names are assigned based on domain knowledge so how to give custom description
They have decommissioned google palm now what to do
I’m facing an error also with google palm what can be alternative do yoh have any idea?
@@kashifahmad9880 It's very difficult to select any other model because other model are not able to Make Sql connections
facing problems in embeddings = HuggingFaceEmbeddings(model_name='sentence-transformers/all-MiniLM-L6-v2')
did u find its solution
Hi, how I will decide that which GPT version will suitable for my project as below code? Please send any guideline/rule regarding this. Thank you.
chatllm=ChatOpenAI(openai_api_key=os.environ["OPEN_API_KEY"],temperature=0.6,model='gpt-3.5-turbo')
Thank you 💯❤
I've been facing a problem where I'm not been able to connect local host. Please help if anyone can.
Amazing video💪
@ 14:33 while executing this db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True) i am facing an error ValidationError: 1 validation error for SQLDatabaseChain
database
instance of SQLDatabase expected (type=type_error.arbitrary_type; expected_arbitrary_type=SQLDatabase) please someone help me
Not working this code now
@@shahrukhahmad4127
@lakshmikanth1988
Could you resolve the error?
How do we add memory to this?
Great 🎉
Thank you !
How to use Gemini?
Great video Dhaval❤
Glad you liked it!
OSError: [WinError 126] The specified module could not be found. Error loading "C:\Users\Moin\anaconda3\Lib\site-packages\torch\lib\fbgemm.dll" or one of its dependencies.
i m getting this error, not able to tackle
❤❤❤
Does this project include any paid Suprciptions for keys?
No
what if the quantity of t-shirt decreases which means someone buys after some time passed, do we need to change few_shots list's values and rerun the whole process? or llm figure out by its own?
Exactly my doubt
You did this project?
Which python version is used in this project & Why? There are new version available like 3.10, 3.11, 3.12
use 3.11 or 3.10 dont go for 3.12
Thanks @@memesthatifoundonreddit6432
I am watching videos on Generative AI from last 7-8 days and this one is so easy to understand. Thanks
Bro first line of code hi run nhi ho rha h pleade help kr do mera final year project h mera 😢😢😢
Facing error at
vectorstore = Chroma.from_texts(to_vectorize, embeddings, metadatas=few_shots)
ValueError: Expected EmbeddingFunction._call_ to have the following signature: odict_keys(['self', 'input']), got odict_keys(['self', 'args', 'kwargs'])
Re-check this things:
1. from langchain.vectorstores import Chroma this is correct
2. formate of your few_shots dict it correct.
3. to_vectorize = [" ".join(example.values()) for example in few_shots] This is correct.
Might solves the issue
I am not able to run, please help me, this is my final year project 😢😢😢 If anyone can help me, I am ready for paying.
Thank you uploading such detailed project videos. Learining newer tech like langchain becomes easier with such tutorials.
Bro first line of code hi run nhi ho rha h pleade help kr do mera final year project h mera 😢😢😢
Thank you for this video! I was wondering if there is a way to do this with Gemini 1.5 Flash or Gemini 1.5 Pro
Your body language is excellent, folk! That makes the topic to understand easily!
Hey, Great Video ... How can we do this for Question Answering LLM Task.
In production answers might change every day, it will be not possible to change answers every day right? And what about if I need data in table format, It will only answer in one single word, if we need data of 2 or more columns, so what would be the approach for it?
Same doubt
Can i write this code in vscode?
Hi sir good afternoon
I am getting error at the starting stage itself while using the google palm
Activate your api key
Amazing project
Error of this line :--to_vectorize = [" ".join(example.values()) for example in few_shots]
This is the error :----------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In[31], line 1
----> 1 to_vectorize = [" ".join(example.values()) for example in few_shots]
Cell In[31], line 1
----> 1 to_vectorize = [" ".join(example.values()) for example in few_shots]
TypeError: sequence item 3: expected str instance, dict found
If I change above line with this to_vectorize = [{" ".join(str(value) for value in example.values() if isinstance(value, str))} for example in few_shots] this is working fine but in next line
vectorstore = Chroma.from_texts(to_vectorize, embeddings, metadatas=few_shots)
it is giving me this error 86 texts = list(map(lambda x: x.replace("
", " "), texts))
87 if self.multi_process:
88 pool = self.client.start_multi_process_pool()
AttributeError: 'set' object has no attribute 'replace'
I resolved it using the following changes:-
in few_shots, make sure to take only the results from the qns1, qns2, qns3, qns4, qns5 .
i;e., qns4['result'] in-place of qns4 and so on.
and same for others.( if qns is alread a string value and not dictionary, then leave it.)
@@prakashraushan2621 did u get a working project?
@@deepashenoy2301 yes it worked for me.
Hi , this video was super helpful for my work but can we retieve the SQL query without running it, actually I want only the SQL Query. I tried with logger but failed. Any good idea would help a lot.
I have no science background in my school level and i completed my post graduation.is it eassy to shift data science filed.. If i can learn data science certificate programm.. If data science short term course are available in data science filed
I am trying to join the list but i am getting error like this "sequence item 3: expected str instance, dict found" how can i solve this error
please help me to comeout from this
Can we access MongoDB as well?, becoz i could not find docs related to MongoDB.
can you make a video on finetuning llm for cyber security.
Can I do this project using Llama or any open source model?
Yes. Please check the latest video on this channel. It uses llama3.1, you can change this project as per that video.
@@codebasics as the result in the browser I get this output "Question: what is the quantity of red tshirt? SQLQuery: SELECT sum(stock_quantity) as total_quantity FROM t_shirts WHERE color = 'Red'" not get output like you
@@arshahin9788 @codebasics I am getting the same problem
Hii sir, 2nd year cse student sir, certification ka bhi workshop rakhiye if possible please
Please make a video on prompt engineering
Feel bad for Lauki Lal
I binged on the playlist, how come this does not have millions of views?
Thanks for this tutorial !
Amazing tutorial. really helpful. Thanks Dhaval Patel sir.
Sir you are using a GPT api key internally how they work and what is providing in the code when you use the chain function then you are not going to encode query. And you pass LLM = LLM means what??
Bro you got a long way to go
@@johannanderson848 means