CHATGPT For WEBSITES: Custom ChatBOT: LangChain Tutorial
HTML-код
- Опубликовано: 22 авг 2024
- In this video, we will create a chatbot based on GPT-3 powered Natural Question Answering BOT on any Website using LangChain. We will combine the concepts of Embedding, Information Retrieval and VectorStore to create a powerful information retrieval system for your website.
LINKS:
Google Colab: colab.research...
OpenAI API Pricing: openai.com/pri...
OpenAI Models: platform.opena...
FAISS: github.com/fac...
Links to Videos:
PDF Files: • ChatGPT for YOUR OWN P...
Open Tools: • Talk to YOUR DATA with...
-------------------------------------------------
☕ Buy me a Coffee: ko-fi.com/prom...
Join the Patreon: patreon.com/PromptEngineering
-------------------------------------------------
Langchain is a powerful open-source Python (and Javascript) framework that allows you to build applications with LLMs and text embeddings. With Langchain, you can easily train GPT on your own data and create a personalized LLM that can answer questions and generate text.
In this tutorial, we will walk you through the process of building an application with Langchain using Streamlit. We will dive into Langchain and its features, including its ability to train GPT on documents and text embeddings.
We will show you how to create a Text Embedding model for your application using Langchain and how to integrate it into your project. You will also learn how to train GPT on PDF documents and fine-tune it to your specific use case.
Throughout the tutorial, we will be building a fully functional application that allows you to upload a PDF file, ask questions about it directly, and have an LLM answer for you. We will use the powerful capabilities of Langchain to create a seamless user experience and showcase the framework's many possibilities.
Langchain is a versatile framework that can be used in many applications, from chatbots to document analysis. We will show you how to leverage Langchain to create powerful applications with Streamlit that can help automate tasks and improve efficiency.
If you're looking to take your natural language processing skills to the next level, this tutorial is a must-watch. Learn how to build powerful applications with Langchain and chat with GPT about your PDF files. So, what are you waiting for? Let's get started with Langchain!
All Interesting Videos:
Everything LangChain: • LangChain
Everything LLM: • Large Language Models
Everything Midjourney: • MidJourney Tutorials
AI Image Generation: • AI Image Generation Tu...
Want to connect?
💼Consulting: calendly.com/engineerprompt/consulting-call
🦾 Discord: discord.com/invite/t4eYQRUcXB
☕ Buy me a Coffee: ko-fi.com/promptengineering
|🔴 Join Patreon: Patreon.com/PromptEngineering
▶ Subscribe: www.youtube.com/@engineerprompt?sub_confirmation=1
This was a game-changer for me. I'm pretty new to langchain and was having a hard time wrapping my head around it, but your video made everything click. Thanks for sharing your wisdom with the rest of us!
Glad it's helpful. Thank you 😊
My friend, I watch most of your videos and I am learning a lot. Your explanations are f******* CLEAR and you show complex things in a way that any one (even I) can understand without any doubt!!!
Thanks a lot for your work, your videos and patience!
Go on! You make the difference!
Thanks :)
Great video, I really appreciate that you kept going back to the diagram while writing the code.
Probably the most clear explanation of this. Thanks. Next up, some UI for it 😊
Soon :)
I was searching or hours for a video like this. Thanks! It's pretty straightforward, and your diagram helps a lot in understanding the concepts
This video explanation is just great.
I was switching between the websites of different products, like LangChain and QDrant, but I was not able to get the overall picture of such a solution.
Thanks for the explanation! It's crystal clear!
Cool. Would be fun to make a website where the nav is entirely handled through conversation with Chatbot
Wow, what a clear and concise explanation!!! Thanks for this and keep on giving us this valuable information.
It is truly amazing. Greetings from China.
Thank you for sharing with the world. Your work is packed with information and remains accessible.
Thanks :)
thank you so much for this video brother. I learnt a lot.
You're awesome. This is so helpful.
Thanks :)
Love your videos, keep up the great work!
Hi, Thank you for this video. Your explanation is great and really helpful in understanding the concepts.
That's really a knowledgable content.
Great video, a good follow-up might include how to use agents to enhance the chatbot for sites that also have dynamic content
very well explained. Thank you!
Awesome explanation, very easy to understand. Thank you
Glad it was helpful!
Great explanation, thank you!
Very good content as usual, two points in case you can comment or consider for the future:
1. Just on this context of web-sites (there is many more data to consider), there is normally plenty of info in web-sites that is provided as non-text, think of diagrams or screenshots, etc, in fact some websites are strongly supported by this type of "image" support, I guess we will need multi-modal support for this at some point in time (you can also have links to documents or other type of data of course) , so asking in case there is anything that could help with this part, for a more comprehensive/relevant scraping of sites.
2. Then the point with secured areas of web-sites, and with this the fact that users may have rights to access parts of a site (or not), think of Confluence web documents that are in private domains with a complex set of access rights, any recommendations as to how to possibly handle this type of data? I guess what I am saying is complex, since we would need to implement Auth access management into what embeddings can be accessed, etc
Thanks for kind words.
1. You are right, most of the LLMs focus on text. There are a few models that have the ability to understand images (think Llava, GPT4-mini, I have videos on both) but at the moment they are computational infeasible to run as part of such a system.
2. Yes, for the propriety dataset, you will need authentication mechanisms. I believe there are some packages which let's you do that.
Splendid! Very well explained.
Glad it was helpful!
Thank you for this great content.
One suggestion is to compare the FAISS with other vector db solutions like Pinecone, etc.
Interesting idea, will do :)
I really really enjoyed your video tutorials. Insightful and no bullshit. And I like your explanation with diagram . Could you share these diagram too? 🥰
Thank you, it will be part of an upcoming course.
@@engineerprompt, Thank you. Will be waiting.
i had to install a lot of other libraries such as:
#!pip install unstructured
#!pip install pdf2image
#!pip install pytesseract
#!pip install tiktoken
also the very last step when asking a question doesnt work for me. it says InvalidRequestError: you must provide a model parameter
could you please help me to resolve this? thank you very much for your video!
You explain very well
Amazing tutorial! I wanted to ask, do you also have a video where you build a Chat UI (in Python)
@engineerprompt Please link if you have a video on this. Thanks.
Thanks great tutorial. Can you do some basic python ai setup tutorials explaining the functions?
OpenAI Chatgpt is able to keep memory of the previous questions, in this tutorial u have used one of their APIs, my question is did u try to check if the chatbot that u created using langchain is able to keep track of your previous questions and it taked that in consideration while answering a new one.?
Yes, you can implement that as part of the langchain. For this specific example I didn't cover it but will be doing it in new videos.
Also what are the differences between openAi embedding models and other open source embedding models? Is it just faster or will I get completely different results?
Can you provide link to excalidraw sheme please?
Looks great
Thanks for sharing
Best teacher ever :) Just to make sure I understood, these embeddings with langchain allow to retrieve the most relevant website parts to be pasted as "context" with the question for the chatbot to answer, right?
Glad you found it helpful.
Yes, the chatbot generates answers just by looking at the most relevant parts and doesn't see the rest.
I need to login into the website before getting access to all the urls documents. How do I achieve that using Selenium URL Loader? Or any other way?
great, thank u
good content. code needs some changes due to updation, but great explanation
thanks, its a really old video, probably need to start creating newer versions of the same :)
this is an amazing video!
Hi, good tutorials
I'm having a little trouble, at the very last step when the user asks a question I get the error :
InvalidRequestError: This model's maximum context length is 2049 tokens, however you requested 2893 tokens (2637 in your prompt; 256 for the completion). Please reduce your prompt; or completion length.
How can I bypass this?
I really appreciate the tutorial but it doesn't help much doing them in Colab. It would be much more valuable if you did them in VSC or some text editor installing the packages. For example UnstructuredURLLoader doesn't work at all, there is a "libmagic is unavailable but assists in filetype detection on file-like objects" error that no one seems to have been able to fix. I appreciate if someone has the solution.
would you please give us link to the diagram you use for this tutorial
I keep getting this error despite creating a brand new OpenAI account. I'm trying to create a chatbot that can read from a pdf file. How do I fix this error: "You exceeded your current quota, please check your plan and billing details"
There's no way I can exceed if I literally just created my account...
Most probably you don't have money in your account.
How will the implementation shared in Google colab change when new contents appear on the website (like a blog). The entire embedding (training data) needs to be run again and the pickle file regenerated. Is that correct? What approach is better if I want to make a model for changing data (like a blog)
Hi bro , kindly do a same thing in open source model kindly do the needful.
Thankyou so much
Hope you do an update video in the future were the ChatBot is actually on a live website answering questions. Or Can you please provide hints on how someone can embed the chatbot let's say on a wordpress website?
Great video. I have been working with langchain for some time now but still this video helped me a lot to fully understand the details. Just one question though! How can we make this work if the data on the website is constantly changing? For example, in the case of an e-commerce website. Do you have any video on that too?
nice video Thanks so much
can you please also create a video on how to host this on an environment and not local so that other people can query it
it's amazing but something is missing , as this should be a chatbot for a Website , so how to deploy this on a website ?
how would you go about hooking this app to a UI?
Also curious about this, please!
soon :)
Great video. Would be amazing if you add how one can create a UI for this chat bot
Check this out for some ideas: ruclips.net/video/RIWbalZ7sTo/видео.html
The only problem is that it also gives responses to out of context questions too. Suppose I have a financial firm but someone come and ask what are the upcoming movies in 2023? It will answer that too. How to prevent it?
Amazing tutorial, Thank you! By following your tutorial, I was able to develop QA bot that can provide answers from a given document. However, this chatbot is limited to the document’s information only. I want to create a chatbot that can integrate its own data with the document’s data. How can I achieve this? Please help!
Thank you! Your instruction is very clear and great. Can you please share the code?
Awesome video!
However, I am curious as to how to store the conversation history.
So that the user can ask follow up questions about the previous response.
Check this out: ruclips.net/video/dxO6pzlgJiY/видео.html
Thanks for your efforts
If you can tell me how use openai embedding with gpt4all
Please share with me any guide for this
will be realizing a video on it soon.
This is a great content .. Thank you. Can you please create a detailed contents regarding execution of BI level queries on top of a Pandas/Spark dataframe using embeddings and models locally (no OpenAI I mean)
thanks, yeah will be covering this in future.
Thanks!
Excellent! Will you explore one or more possible web UIs like Gradio or Streamlt?
Yes, working on a video, will be making a whole series. :-)
In the scenario of conversational robots, how to limit the token consumption of the entire conversation?
For example, once the consumption reaches 1,000, it will prompt that the tokens for this conversation have been used up.
i am geeting error [Document(page_content='Access Denied
You don\'t have permission to access
i have used web based loader no error is comeing but useing UnstructuredURLLoader getting error .
can please difference b/w web based loader and UnstructuredURLLoader
This was good, but the thing I wanted to learn is how to use the site map to feed all of the urls for a website into a document loader using code? Can you provide a code snippet by chance? Thanks!
When i run this code, the "source" is showing as " " (empty string). Can someone let me know why the source url is not showing?
Thank you for your time in this video. Could you clarify the following for us?: with this set up, does the model look for answers both using Gpt-3.5 general knowledge (same as chatgpt) AND documents from langchain? Or only documents from langchain and not general chatgpt knowledge?
With this, it will primarily use the provided information for answering along with ChatGPT training data. You can limit that by providing a prompt that asks the model to use the provided context (information). Hope this helps
Discord would be great.
Talkin to me?
@@fACEB0T Why do you think so?
I finally found time to set it up. Still learning the ins and out of it. Hopefully will be sharing this week. Thanks 😊
@@engineerprompt Dont worry, I can help.
Is there a bug in the vicuña response? The new line characters are inside the source url. ¿Is that avoidable or that output is generated by the llm ?
Thank you, very useful video. How remember the previous chats and consider the new response?
I like this kind of tutorial with a flowchart to following the steps easily.
I saw you use vs code with a Jupyter Note plugin? I this approach better or can I use Jupyter notebook separately ?
Hopefully more from this stuff. Many thanks
Also interested how you did this notebook style in vscode
VScode has a plugin for notebook. I usually use notebooks for experimentation but for production, I use scripts. I personally find it helpful to have everything in a single place. But ultimately its just personal choice :)
there is a plugin for it. You can search for it.
@@engineerprompt Thx for the feedback.
What about all the other sources that are in the website such as documents and pdfs?
Is it just plain text we are talking about here?
Thanks alot for sharing this video with us...
But there is a problem that the default model is working but gpt-3.5-turbo is not working... Any solution for this... please guide... I'll appreciate that...
No. Here is the problem, a lot of Chatbot I can't customize the Avatar. I can't customize the "look" on the lower right "avatar" when launched, I can't customize anything except data input. This is sucks. I try all 10 diff Chatbots none can do what I want to do and look and feel...all chatbot looks "stupid" i cant to have my own look with a message so visitor will click.Plus the chatbot reply I want to emulate OPEN Ai in a slow motion and not a photo graphic snap shot reply. It does not emulate human motion. Anyways...
Great video but I have a question for you sir.. Can you integrate chat GPT into a website that already has chat systems between users and then integrate chat GPT into one fake profile to act like a chatter and train it to chat like normal person ?
I have a very simple question. TO use Open ai embiddings model lets say ada-002 do we need subscription?
Is it possible to have a chatbot like Chatgpt in a website that was trained and ready to answer any question the person might have
How to remove unwanted data from vector file using pattern matching or any other method.
Greetings sir, I am really inspired by you thanks for your work, I would like to know that how can I use vicuna, alpaca or Llama instead of Openai to reduce the cos. Really appreciating your efforts my mentor :)
would b good if u included ur ipynb file as well
I tried to use this for my website but it was not working. Can you assist me somehow?
seria bueno que comiences a aplicarle la api de ingenieria inversa para abaratar costos, ya que utiliza el modelo gratuito de OpenAI
My code is given this error
AttributeError: module 'openai' has no attribute 'Embedding'
Some one could help me? I already tried Python versions 3.10 and 3.11
I think the LangChain implementation has changed. I will try to rectreate this with updated code
Thanks
Thank you!!!
Excellent video . this video where i learn so much about langchain. i have question that i have make chatbot for house searching and i store dataset data into pinecone . data like location, bhk, sqft, price now our chatbot is only answering the question whatever he asked but client want the if we cannot provide such information like we cannot provide location or bhk, then bot will ask question about that. can you please tell how i can make this type of chabot?.
Thank you, I think you will need to create two layers, the first one check for any missing information (with memory) and asks user for it. Then you can have this second layer which takes the updated input and use it for the information retrieval.
Do you have a tutorial on how to take this model online for a Wordpress website?
Hi thanks for the video, just wondering why it doesn't support If I create this code for python environment?
The code sample you shared doesn't work. Please check and see if you can update Colab notebook.
What's the error? I runs on my end. You will need to provide your own API key.
How is it different, now that we can customize ChatGPTs?
Hi, how to use LangChain in multiple sources pdfs, txt, web pages, JSON all this in one knowledge base? Will you create that video tutorial?
how to do web scraping chatbot from URL
Great work dude. Is it possible to use use Vicuna or other instead of openIA ? Thank you
Yes you can! check this out: ruclips.net/video/MlyoObdIHyo/видео.html
Excellent video!! i have to create a chatbot that only answer questions about the data from a specific client, for example store location, hours, stock, etc. I need to give the bot a kind of personality, not only answer questions, but also respond with a certain "charisma", how can I do that? I thank you for all the information
Look at adding a system prompt. That way, you will be able to give it a persona.
@@engineerprompt thanks for that!! do you know where i can learn how to adding a system prompt? Thanks again!!
What if you want the model to return paragraphs as source material instead of the urls? Is there a way to make it return the specific text chunks that is got it’s answer from?
Yes, you can easily do that. You will have to use a different qa_chian in langchain. Will cover this in another video.
Many thanks, what about using Redis do you have code.
will this langchain library will also work for dynamic webpages?
module not found langchain. How do i solve this
Hey Hi awesome video.
I have very basic coding skills and was using the colab file you have provided. However when running the chain command I am getting an error " f"OpenAIChat currently only supports single prompt, got {prompts}"
CAn you please help
LangFlow Tutorial pls
didn't know about it. Will check it out. Thanks for brining it to my attention.
do you have a video where the same task can be done, but instead of OpenAI models, using Falcon?
Check out my localgpt videos
One suggestion:
Use light theme for teaching purpose. White gives more clarity and visibility.
You can compare both(black and white) and validate.
No. Light theme sucks. Dark is pretty good.
I prefer dark
@@lovol2 light hurts the eye...eeew
"unstructured package not found" is the error while i am importing UnstructuredURLLoader from langchain module
You probably need to install that separately as well.
@@engineerprompt i did but still the same error