Cannot thank you enough Greg! Your efforts to simplify these concepts are highly appreciated. I learned a lot from you. This specific video is a perfect second episode of the first Cookbook! Thank you so much.
In the evaluation section, the answer input was “Healthkit”. Very cool that langchain immediately recognized the “Heathkit” had been misspelled in the answer.
Thanks again for another great video! I'm really enjoying learning about all the different things I can do with LangChain. I spent the long weekend here in Thailand catching up on your videos and following your coding instructions. I've already finished this one (haha, I had to skip a few videos that required an OpenAI key that I don't have yet). I'm definitely going to keep watching the rest of your playlist. Thanks again for all your help!
As you said, now up to build. Always can come back to this video for a quick refresher on a specific use case. Great to have your content as a guideline. Thank you for your work.
Great vids, Greg. A bit overwhelming for me still but slowly getting there. And even bigger thanks for introducing me to Wild Belle. What a great band!!!
Thank you very much, your videos have encouraged me, a Vietnamese person, from a zero level of LLMs, LangChain, and its ecosystem to a beginner in LLMs. I implemented some of your lessons in my apps, and get incredible results.
Hey Greg, Greetings from Ireland. I just want to say how great your videos are for a beginner like me. I had no idea what AI, Machine Learning or Langchain was a few months ago and now I'm obsessed. I'm put in 4-5 hours every day following videos like yours and trying to create stuff. Keep it up please!
Your video was very informative, and I appreciate it. The primary challenge that many individuals are facing is getting the LLM model to provide references as well. This is particularly crucial for Question & Answer tasks, as it's important to understand how the answer was generated and which text was used to generate it. Have you experimented with this aspect?
I agree that determining the source of answers is key for productive use of AI for Q/A applications. On what authority is the truthfulness of the answer. We make better decisions when our information is true.
Thank you for the brief explanation of the use cases. Can you please make a video on querying the website or in other words build a chatbot on any website using LangChain.
Brilliant vid. Everyone is focusing on langchain with OpenAI. I am interested in using agents, chatbots, and memory with local LLM’s. Not sure which of the newer instruction following models are best, and how the limitations of smaller models affect decisions around building with langchain.
Great tutorial! We need to pip install openai, langchain and tiktoken, faiss-cpu and the similar files uploaded to notebook used in the demos. I finally got my huge RUclips transcript to work thanks to you! I was using the same langchain stuff but for some reason it was throwing an error during similarity search. I figured out that the charactersplitter isn't splitting my file so i had to use texttoken splitter for some reason.
It would be cool if you could show a tutorial where you use open-source LLM models like those from Hugging Face instead of OpenAI. The project I would be most interested in would be one where you could use the open-source model for agents, specifically to see if a question needs more information, and maybe even have it be able to choose between querying a local database and/or looking it up online.
Hi Greg, thank you so much for your very insightful explanation of Langchain! It would be interesting an introduction on how you design an application using this fantastic tool. There is a combination of code, langchain (prompt, agents, tools, chains, etc.), "chat", data to put together and it is not the usual way of thinking an application.
good vid greg. Ive been experimenting with the different text splitters, i found NLTKTextSplitter was good when vectorising all of langchain docs, as there is text + python code in there. there is also a pythontextsplitter() which looks promising.
Hi Greg, these are very helpful tutorial vids, much appreciated! One thing I cannot find any information on in whatever example / turorial / documentation I find is about the memory / chat history. Thanks to your tuts I created a chat app very quickly, which stores the chat history (tested with the 'What was the first thing I asked you about'. But, since I will be creating an app with this later, is that information stored per user? As in: per session? Because the Python app is a single instance, what happens if [n] users all talk to that application? Do you know if langchain automagically separates the history per user session? Thanks in advance!
Suppose if I save the whole source code of a laravel project as a document ,will langchain help me get total number of controllers used in the project ?
Hi Greg, I have been watching your excellent video and trying out things. It save me so much time. All in one place. Many many thanks. Can you help me with a problem with 8.40 when I run qa.run(query). It threw an error. AuthenticationError: No API key provided.
Eh, I found that it reduces hallucinations. Usually my summaries are for function rather than form so I don't need to much fluff in there. Either way I would encourage you to experiment with your own use case and see what you like
I've seen your videos along with others like them, and I've noticed that most of you use notebooks for your demonstrations. As a result, the front-end web interface, which plays a crucial role in showcasing the results, is often not discussed. This includes important aspects like collecting data and sending it to the models, as well as displaying the output in an effective way on the front-end, especially when using chains. It would be helpful to know which frameworks are best suited for handling these front-end tasks.
@@DataIndependent Question and answer, text summarization, and chain dialogue are valuable tools for addressing queries in large documents. However, they have limitations, like not being able to identify the specific page number or other metadata related to the answer. For example, if I receive an answer based on a question from an embedded document, I would appreciate being directed to the relevant section of the document to explore further.
@@JTedam Can't you take the context that is being provided to the LLM based on semantic search on the query, and do the code version of Ctrl+F of that context in your document?
Hi, Thank you for your tutorials. I am following your tutorials for quite some time now. I have watched your whole playlist on this. However I am unable to figure out best economic approach for my use case. I want to create a Q & A chatbot on streamlit which answers only my custom single document of about 500 pages. The document is final and won't change. From my understanding so far, I should either choose Langchain or LlamaIndex. But, I will have to use OpenAI api to get best answers, but that API is quite costly for me. So far I have thought of using Chroma for embedding and somhow storing the vectors as pkl or json on streamlit itself for re-use, so I don't have to spend again for vectors/indexing. I don't have enough credits to test different methods myself. Kindly guide me. Thank you.
Hi Greg.. thanks so much for sharing your expertise! Super helpful.. had a question - looks like with LangChain + OpenAI, you can pretty much extend and interact with any data source as an extension to the OpenAI LLM. In what scenario would you want to create a model that's trained on the external data vs just using this method of extending with LangChain?
6:11 in all examples I’ve seen (and this one) Q&A document retrieving relies on the text content. How can you impose a semantic structure on chunks, like all chunks in chapter 3 have that metadata, or an indicator in the vector embedding. Basically, I want the retriever to get all chunks within a section regardless of text content. Any ideas?
Great video. One question. I can't import Pinecone with error message like this: ModuleNotFoundError: No module named 'pinecone' C:\Users\Administrator\tool\Topic Modeling>pip install pinecone ERROR: Could not find a version that satisfies the requirement pinecone (from versions: none) ERROR: No matching distribution found for pinecone Could you pls help me on this? Thanks!
For QA over documents, does OpenAI charge for only the query but not the context? Is it more effective to store as much as possible in context rather than the query. It may not be possible for most applications but for mine this would possible.
did you ask chatgpt why you are getting this error? I have found when I post in an error message to chatgpt it has been very good at giving me ideas of what could be wrong and how to fix it.
I've been experimenting with the "interacting with api" section but I noticed it's very expensive. $0.1 for every call you do. Are there any ways to reduce the OpenAI costs?
Thanks. been following your posts. Great. Question: it seems that when using langchains way to Q&A documents with ChatOpenAI it answers also q outside the context of the documents. the outdated conversationalretrievalchain did work well. Any thoughts?
Totally! You can just ask it to classify your text. Whatever you’d like. However it’s an expensive classification engine and best for ambiguous or grey area text. For more traditional data science classification there are better algorithms
so this so called gentleman author replies only to compliments and appreciations, not for serious comments. hello author, let go of your ego and you will learn only when you respond to criticisms and to the problems reported by the watchers. not for those YES saying sheeps. hope that goes into your grey matter
Hey thanks! Is do you have a comment I can respond to? The volume of comments gets overwhelming so I don’t have time to respond to them all. When you create content you do it for fun and I need to keep a watch about what activities give me energy. I find that responding to trolls or haters doesn’t give me energy. Responding to happy or helpful people with constructive criticism does. Thank you!
It’s a balance of attention span vs effectiveness. I don’t think I can please 100% of people (that shouldn’t be the goal either). So far this of the first comment I’ve had on speed. Also, it’s easier to press the pause button than it is to speed up and slow down the playback speed constantly. So all that to say, thanks for the comment and the feedback I love the energy
this is another one - when I tried the next API : ValidationError: 1 validation error for APIChain __root__ Please provide a list of domains to limit access using `limit_to_domains`. (type=value_error)
I am following your tutorial on chatbot 20:18. I am getting an error. You are trying to use a chat model. This way of initializing it is no longer supported. Instead, please use: `from langchain.chat_models import ChatOpenAI`. I imported this but I do not know how to initialise it?
Thank you for the thorough walkthrough, it's very inspiring! Perhaps for future video, would you be interested to try work with some offline models, embedding (Milvus) and working with graph database?
if you are objective is for us to try your code, then make sure it works you know. what is the point of trying something and if it throws error after error. this is really annoying
Thanks for making this. One suggestion! When you tap on your table and hit your enter key, the sound travels to the mic and it's not a pleasant sound. As you probably already know, sound is really important to the user experience. A lot of people say that it's even more important than the video quality. Thanks again!
Greg, you and James Briggs became my favorite go-to-channels to learn Langchain. High level stuff!
Nice!! Thank you
Cannot thank you enough Greg! Your efforts to simplify these concepts are highly appreciated. I learned a lot from you.
This specific video is a perfect second episode of the first Cookbook!
Thank you so much.
This guy is the GOAT.
Wow thank you!
you are absolutely true！
This is by far the best explanations of LangChain I have found. I will follow you next videos closely. Keep up the good work, Greg!
In the evaluation section, the answer input was “Healthkit”. Very cool that langchain immediately recognized the “Heathkit” had been misspelled in the answer.
I am so excited... the possibilities for this are endless
Nice! Glad to hear it - what're the first things you want to build?
Thanks again for another great video! I'm really enjoying learning about all the different things I can do with LangChain. I spent the long weekend here in Thailand catching up on your videos and following your coding instructions. I've already finished this one (haha, I had to skip a few videos that required an OpenAI key that I don't have yet). I'm definitely going to keep watching the rest of your playlist. Thanks again for all your help!
As you said, now up to build. Always can come back to this video for a quick refresher on a specific use case. Great to have your content as a guideline. Thank you for your work.
I just want to say thank you very much. I really like the way you explain how to use LangChain.
Awesome, thank you Port!
16:48 Crazy levels here. As database developer, this is amazing.
Great vids, Greg. A bit overwhelming for me still but slowly getting there. And even bigger thanks for introducing me to Wild Belle. What a great band!!!
Ha - I'm glad you liked them! I was hoping someone on watching this would introduce me to Natalie but not yet
Really useful video! Thank you for taking the time to record it!
Glad it was helpful!
Thanks Greg! Enjoy plenty of health and business so that you can continue providing valuable content.
Doing Gods work. Thank you for this!
Thank you very much, your videos have encouraged me, a Vietnamese person, from a zero level of LLMs, LangChain, and its ecosystem to a beginner in LLMs. I implemented some of your lessons in my apps, and get incredible results.
One mistake: the correct answer is not Healthkit. It is Heathkit.
Yeah, I had one of those and came in down here to post that. Thanks.
Hey Greg, Greetings from Ireland. I just want to say how great your videos are for a beginner like me. I had no idea what AI, Machine Learning or Langchain was a few months ago and now I'm obsessed. I'm put in 4-5 hours every day following videos like yours and trying to create stuff. Keep it up please!
Me too - Almost a new career!
Nice! Thank you to both of you.
Please make sure to share your work with the community
Great explanation - Just one blip - s/n ratio reduction. It happens to all of us 😊
Great explanation !
Just one blip - s/n ratio reduction. It happens to all of us 😊
Cleared my doubts and developing some Insights as well.
Thanks again - glad to hear it's going well
thank you for sharing. did not know that langchain has this many use cases. amazing
Awesome! Glad to help Gurmukh!
Great work! Your video provides a thorough explanation.
Thanks Jay
Thanks for all these great videos! Amazing job, you make the complex feel simple! Excited to start building!
Thanks so much Greg. I’ve been hanging out for this one since the release of part 1. It didn’t disappoint - Legend!
Awesome man. Been waiting for a couple days now. Excited to follow along.
These are great videos. I appreciate your mission to aid in an individuals ability for data independence. Keep it up good sir!
Nice! Thank you Christian
Great video! Thank you very much
Awesome thanks Brian
This was extremely helpful Greg thanks so much!
Awesome! Thanks Jaime
Thank you so much! Was a fun time working through these with you, i learned a ton. Great job!
Great stuff! I'm going to explore this in depth this week, you saved me quite a bunch.
Nice! What do you want to build?
Thank you for providing such excellent resources.
you and your content both are awesome 💖
Your video was very informative, and I appreciate it. The primary challenge that many individuals are facing is getting the LLM model to provide references as well. This is particularly crucial for Question & Answer tasks, as it's important to understand how the answer was generated and which text was used to generate it. Have you experimented with this aspect?
I agree that determining the source of answers is key for productive use of AI for Q/A applications. On what authority is the truthfulness of the answer. We make better decisions when our information is true.
Very informative. Thanks for posting
Thank you for the brief explanation of the use cases. Can you please make a video on querying the website or in other words build a chatbot on any website using LangChain.
Next level Greg!! Outstanding!
Bets content out there.
Greg is the GOAT 🐐
Brilliant vid. Everyone is focusing on langchain with OpenAI. I am interested in using agents, chatbots, and memory with local LLM’s. Not sure which of the newer instruction following models are best, and how the limitations of smaller models affect decisions around building with langchain.
Another great video! Thanks!
Great tutorial! We need to pip install openai, langchain and tiktoken, faiss-cpu and the similar files uploaded to notebook used in the demos. I finally got my huge RUclips transcript to work thanks to you! I was using the same langchain stuff but for some reason it was throwing an error during similarity search. I figured out that the charactersplitter isn't splitting my file so i had to use texttoken splitter for some reason.
these are really well structured and presented.
fantastic!
Yorkville is a neighborhood in New York City, just saying. Great video!
Dude you rock!
It would be cool if you could show a tutorial where you use open-source LLM models like those from Hugging Face instead of OpenAI. The project I would be most interested in would be one where you could use the open-source model for agents, specifically to see if a question needs more information, and maybe even have it be able to choose between querying a local database and/or looking it up online.
Great lectures.
Hi Greg, thank you so much for your very insightful explanation of Langchain!
It would be interesting an introduction on how you design an application using this fantastic tool. There is a combination of code, langchain (prompt, agents, tools, chains, etc.), "chat", data to put together and it is not the usual way of thinking an application.
brilliant!
That's some solid content
Amazing ❤
Thank you! 🙏
Thank you for your contributions to this space 🙏 Which tool are you referring to, called "Metal". Visible as a comment in your notebook at 8:40
great
Thanks for all the info.
What technical knowledge a non-technical person should acquires to start utilizing this?
Change
model='gpt-3.5-turbo' -> model_name='gpt-3.5-turbo'
good vid greg. Ive been experimenting with the different text splitters, i found NLTKTextSplitter was good when vectorising all of langchain docs, as there is text + python code in there. there is also a pythontextsplitter() which looks promising.
Hi Greg, these are very helpful tutorial vids, much appreciated!
One thing I cannot find any information on in whatever example / turorial / documentation I find is about the memory / chat history. Thanks to your tuts I created a chat app very quickly, which stores the chat history (tested with the 'What was the first thing I asked you about'. But, since I will be creating an app with this later, is that information stored per user? As in: per session? Because the Python app is a single instance, what happens if [n] users all talk to that application? Do you know if langchain automagically separates the history per user session? Thanks in advance!
Suppose if I save the whole source code of a laravel project as a document ,will langchain help me get total number of controllers used in the project ?
Excellent sessions. Can we do all the tutorials in CPU laptop or do we need GPU laptop?
There isn’t a lot of computation happening so you shouldn’t need a special set up
Hi Greg, I have been watching your excellent video and trying out things. It save me so much time. All in one place. Many many thanks. Can you help me with a problem with 8.40 when I run qa.run(query). It threw an error. AuthenticationError: No API key provided.
You need to provide your API key to your script. That should be one of the first code cells at the top of notebook
@@DataIndependent Thanks! I did this and the problem went away os.environ['OPENAI_API_KEY'] = "..."
Presumably the same thing.
Audio Volume is low in part1 and in Part2 both
Great video Greg! At 3:53, I noticed you have temperature 0 for your summary chain. Have you found that's ideal for summaries?
Eh, I found that it reduces hallucinations. Usually my summaries are for function rather than form so I don't need to much fluff in there. Either way I would encourage you to experiment with your own use case and see what you like
@@DataIndependent makes sense. Creativity isn't needed for summaries. Thanks again for your excellent videos!
I've seen your videos along with others like them, and I've noticed that most of you use notebooks for your demonstrations. As a result, the front-end web interface, which plays a crucial role in showcasing the results, is often not discussed. This includes important aspects like collecting data and sending it to the models, as well as displaying the output in an effective way on the front-end, especially when using chains. It would be helpful to know which frameworks are best suited for handling these front-end tasks.
What types of applications are you trying to build?
@@DataIndependent Question and answer, text summarization, and chain dialogue are valuable tools for addressing queries in large documents. However, they have limitations, like not being able to identify the specific page number or other metadata related to the answer. For example, if I receive an answer based on a question from an embedded document, I would appreciate being directed to the relevant section of the document to explore further.
@@JTedam Can't you take the context that is being provided to the LLM based on semantic search on the query, and do the code version of Ctrl+F of that context in your document?
Hi, Thank you for your tutorials. I am following your tutorials for quite some time now. I have watched your whole playlist on this. However I am unable to figure out best economic approach for my use case.
I want to create a Q & A chatbot on streamlit which answers only my custom single document of about 500 pages. The document is final and won't change. From my understanding so far, I should either choose Langchain or LlamaIndex. But, I will have to use OpenAI api to get best answers, but that API is quite costly for me. So far I have thought of using Chroma for embedding and somhow storing the vectors as pkl or json on streamlit itself for re-use, so I don't have to spend again for vectors/indexing. I don't have enough credits to test different methods myself.
Kindly guide me. Thank you.
Can we learn longchain without zero knowledge of phyton???
Hi Greg.. thanks so much for sharing your expertise! Super helpful.. had a question - looks like with LangChain + OpenAI, you can pretty much extend and interact with any data source as an extension to the OpenAI LLM. In what scenario would you want to create a model that's trained on the external data vs just using this method of extending with LangChain?
6:11 in all examples I’ve seen (and this one) Q&A document retrieving relies on the text content. How can you impose a semantic structure on chunks, like all chunks in chapter 3 have that metadata, or an indicator in the vector embedding. Basically, I want the retriever to get all chunks within a section regardless of text content. Any ideas?
Yay!
Great video. One question. I can't import Pinecone with error message like this:
ModuleNotFoundError: No module named 'pinecone'
C:\Users\Administrator\tool\Topic Modeling>pip install pinecone
ERROR: Could not find a version that satisfies the requirement pinecone (from versions: none)
ERROR: No matching distribution found for pinecone
Could you pls help me on this? Thanks!
Sounds like the environment you're in doesn't have pinecone. Have you tried installing it?
For QA over documents, does OpenAI charge for only the query but not the context? Is it more effective to store as much as possible in context rather than the query. It may not be possible for most applications but for mine this would possible.
OpenAI will charge for how many tokens you send over, this includes the query and the context
@@DataIndependent Thank you!
Heathkit, not Healthkit
yes, was it a typo?
As a young lad, I (unsuccessfully) built a Heathkit AM/FM clock radio. It came with a sodering iron, transistors, resistors, etc...
When I try to run the eval under "Vanilla Extraction", it gives me an 'invalid syntax' error.
FYI I'm running this with Python 3.11, not 3.10.
did you ask chatgpt why you are getting this error? I have found when I post in an error message to chatgpt it has been very good at giving me ideas of what could be wrong and how to fix it.
I've been experimenting with the "interacting with api" section but I noticed it's very expensive. $0.1 for every call you do. Are there any ways to reduce the OpenAI costs?
Hi can you share you Jupyter notebook?
Check the description!
Thanks. been following your posts. Great. Question: it seems that when using langchains way to Q&A documents with ChatOpenAI it answers also q outside the context of the documents. the outdated conversationalretrievalchain did work well. Any thoughts?
Hi greg thanks a lot for your videos.. Do you know if we can perform classification task using langchain+gpt ?
Totally! You can just ask it to classify your text. Whatever you’d like.
However it’s an expensive classification engine and best for ambiguous or grey area text.
For more traditional data science classification there are better algorithms
Awesome tutorial, as your other videos. I would suggest you use chroma, instead of FAISS. I couldn't make faiss work on my MacBook M2.
17:20
Is this where you stopped watching?
@@DataIndependent No it's the part I needed for a project 😁
plz avoid this :)
except Exception as e:
pass
you can spend then a lot of time debugging
so this so called gentleman author replies only to compliments and appreciations, not for serious comments. hello author, let go of your ego and you will learn only when you respond to criticisms and to the problems reported by the watchers. not for those YES saying sheeps. hope that goes into your grey matter
Hey thanks! Is do you have a comment I can respond to?
The volume of comments gets overwhelming so I don’t have time to respond to them all.
When you create content you do it for fun and I need to keep a watch about what activities give me energy.
I find that responding to trolls or haters doesn’t give me energy. Responding to happy or helpful people with constructive criticism does.
Thank you!
Also, if you have any content, videos, or material you’d like feedback on please let me know
Tutorial speed is too high, Practically not possible to follow tutorial having speed in narration and videos. Its like keep hearing like story.
It’s a balance of attention span vs effectiveness.
I don’t think I can please 100% of people (that shouldn’t be the goal either). So far this of the first comment I’ve had on speed.
Also, it’s easier to press the pause button than it is to speed up and slow down the playback speed constantly.
So all that to say, thanks for the comment and the feedback I love the energy
really cool video! what is the difference between GOOGLE_API_KEY GOOGLE_CSE_ID and serpapi?
this is another one - when I tried the next API : ValidationError: 1 validation error for APIChain
__root__
Please provide a list of domains to limit access using `limit_to_domains`. (type=value_error)
I am following your tutorial on chatbot 20:18. I am getting an error. You are trying to use a chat model. This way of initializing it is no longer supported. Instead, please use: `from langchain.chat_models import ChatOpenAI`. I imported this but I do not know how to initialise it?
You should be able to use the same "ChatOpenAI" name as you did before. I'll take a look at the code and update it if necessary
Thank you for the thorough walkthrough, it's very inspiring! Perhaps for future video, would you be interested to try work with some offline models, embedding (Milvus) and working with graph database?
Nice, thank you for sharing and I'll add that to the list
if you are objective is for us to try your code, then make sure it works you know. what is the point of trying something and if it throws error after error. this is really annoying
This is all great fun. Won't M$ just leverage their position with OpenAI to bake all this into WIndows?
Rarely is the out of the box solution that built into the desktop exactly what you need. This library is for when you need to build an application
can you show how we can use Cohere instead of OpenAI?
even though everything is in the docs it's cool to have more examples and use cases! Thank you
Thanks for making this. One suggestion!
When you tap on your table and hit your enter key, the sound travels to the mic and it's not a pleasant sound. As you probably already know, sound is really important to the user experience. A lot of people say that it's even more important than the video quality.
Thanks again!
Thanks for that! I'll watch out for it in the future
Please keep making these videos! I've learned so much, it's amazing!