Falcon-7B-Instruct LLM with LangChain Tutorial
HTML-код
- Опубликовано: 5 ноя 2024
- This Tutorial teaches you how to use Falcon LLM with LangChain to build powerful OpenSource AI Apps
Colab - colab.research...
❤️ If you want to support the channel ❤️
Support here:
Patreon - / 1littlecoder
Ko-Fi - ko-fi.com/1lit...
This youtuber is very nice and outperform others because he directly provide colab version for everyone , especially amateur.
Thank you Rayden!
💯 Thanks for a perfect video. The audio is clear, the pace is perfect, the content is timely, accurate, concise and well presented. You, sir, are doing it right.
Thank you for the detailed feedback
Thank you for the perfect studio lighting🤗No flickering!👌
This guy ever has ever the best practical videos. His explanations are cristal clear and he ever shares a notebook! Thanks a lot!!!
Thanks for the positive comment. Means a lot !
Are you entertained :) ?
Hell yeah all the way bro, thanks for quality.
If you continued with this video series, incorporating a document store, saving outputs to drive, perhaps running qlora on a pro+ instance with an a100, you could start a wave of small businesses who all use colab to maintain and develope their personal ai assistants. Google should sponsor your channel, you rock. Very entrepreneurial
Great tutorial. Do you have an idea why it does not work with the 40b model? I get this error: ValueError: Could not load model tiiuae/falcon-40b-instruct with any of the following classes: (, ).
@@broofya3805 on the free GPU or generally ?
@@mr.yayaman Thanks bro!
nice video sir! what about fine tuning the model?!
Great video, thank you so much! I have a question - using the method in your video, does the model actually run locally? So, it does not go to any external API which in turn could provide great safety of your chat history? Is that correct?
It downloads the model and runs locally. You can check the disk space.
Great video, thanks! Waiting for the Q&A application.
Hopefully, Coming soon!
Great video, thank you for this tutorial. have you any tutorial about this model using chroma db and WebBaseLoader to retrieve data from web and save on db?
Your timing on this is impeccable, I really needed this. Also I can't believe how impressive Falcon 7b is, this is crazy!
Glad you liked it! Thanks :)
Great video and great pace! I noticed that running the same question over and over returns different answers, is there a way to enforce it to give the same response every time?
How would you pass the results back to Langchain for further refinement?
Awaited tutorial 😁
Thank you. Thanks to the weekend :)
Thanks for this informative video, I want to get response from LLM in array from. Means I will give one paragraph text and will ask generate mcq questions from above text. Then I question , answer and options in separate array as dictionary. Is it possible sir? Or Do I need to finetune on my own dataset?
This takes a lot of time when I use Falcon with RetrievalQA chain in Langchain. How do I make the inference time Faster?
Thank you ! for the amazing video . Could you make a video for fine tuning the model with personal dataset ?
Really nice reaction when you got that friendly note form Colab. Anyhow, it is good to know that Falcon now can be run with langchain. Do you know whether it is possible to run Falcon using Qlora for inference? I tried it --> while it does run, still it takes way too much VRAM, i.e., without QLora 15GB and with Qlora 15GB. So there must be something wrong here.
😂 yeah, his reaction!!!
Nice video though! I needed this.
Tried the gpt-j model with langchain locally. I would try this too...
Thanks Javed :) I was shocked to see being disconnected while I'm on the notebook, maybe inactivity. It should be possible. I'm trying to work on it next.
*Great tutorial , Thanks 1littlecoder*
Waiting for the next video!!
As soon as possible
Thanks for the tutorial. Can do a tutorial on how to use falcon with documents answering 🙏
I’m working on a game POC for using an open source LLM to make NPCs more autonomous. Is this Falcon model the best choice or would you suggest working with another model. My focus is not so much about chatting as it is about providing the NPC better, spontaneous decision making/actions regarding the game world. Looking to create something closer to what we saw with Minecraft but in a smaller scale involving fewer possible actions.
Everyone's excited about Falcon. So I'd start with it. If you don't find Falcon good, then try one in the Llama family
*I tried Falcon 180b demo on hf , it's good , even I have to try to give more specific prompts to achieve what I need*
lol not on your pc unless you have some power house sever setup!! the 40b takes some massive gpu power my rtx 3080 is starting to look like junk for llm lol
please tell how to get response by using custom data like text files or csv files. In langchain documentation they have given an example by using openAI embeddings. is it possible to do the same using huggingface embeddings?
Thank you so much for your ice explanation, however I am trying to run the notebook for the same code but when I try running the pipeline I keep getting this error
ValueError: The current `device_map` had weights offloaded to the disk. Please provide an `offload_folder` for them. Alternatively, make sure you have `safetensors` installed if the model you are using offers the weights in this format.
any clue how to solve this?
How can I utilize the Falcon model to produce embeddings? Is it possible to combine it with Langchain to generate a vector database and create the embeddings?
Generally, how do I know if a model fits my GPU? I understand there's correlation between number of parameters in a model and its size, but is there a way to know the exact memory requirement?
This depends on the data type of the parameters. bfloat 16 requires 2 Bytes per parameter. So 7B parameters would need at least 14GB to load the model
lets goooooo bro! show must go on!
Thanks for a great intro to LC+falcon!
Trying to prompt it with other inputs (CSV dataframe: "df = pd.read_csv('data.csv')
pd_agent = create_pandas_dataframe_agent(llm, df, verbose=True)" ) failed with "OutputParserException: Could not parse LLM output: `Answer`". Any idea how to resolve?
Your agent is reading the csv file or writing the csv file?
@@aischool0912 I'm loading a csv and simply trying to get the same outcome as the tutorial, only that instead of a template I feed in a CSV
Use csv_loader instead of pd.read_csv
@@aischool0912 thanks - tried that and it didn't work as well. On inference, the model returns "OutputParserException..."
@@arielnoyman These open-source models are not good at generating code. That is why you are getting that error, as in the Pandas agent, you actually generate code in Pandas. But not the case in these opensource models.
Can you also make a proper interface where we can input question and get answer in another text box
Good job, very enjoyable and informative. Are you in Chenai or Bangalore or ?
Bangalore sir.
@@1littlecoder I love Bangalore! I went to Bishop Cottons a few decades ago. Wishing you my best.
@@1littlecoder man i thought you were in usa or something, we need to meet buddy, very literally very few people have knowledge like this at present in India
Sir please show how to use Falcon 7B and vector db to make chatPDF type of document
How would I go about using this code or similar to query a specific dataset I give it, please?
Could you please elaborate your request?
@@1littlecoder I am trying to select a legal dataset from huggingface like ‘super-glue’. Then use Falcon as a way of querying this specific dataset. Do you happen to know how I would go about this please? Any suggestions/advice would be much appreciated, thank you!
In other words, how can I import custom dataset(pdf, csv, txt) into model that has falcon-40b, and then ask/chat with it?@@1littlecoder
Dose someone compare the performance，result and runtime between falcon and openai davinci model？
you're the best !! thx a lot, i learn a lot :D
Thanks so much !
Hii Sir, can you make a tutorial on how to fine tune this model for specific tasks like to text classification and sentiment analysis, ner , etc.
Thank you
Great work sir
Hello! how could i solve this error? ValueError: Could not load model tiiuae/falcon-7b-instruct with any of the following classes: (,).
Instead of gradio or electron for your next video, please consider using streamlit or chainlit, to avoid all the messy javascript stuff.
You are great.
Thank you sir
Can we run Falcon-7b or 40B on hugging face spaces?
Only on GPU
@@1littlecoder I could not find space tiiuae/falcon-40b or 7b to duplicate. Did you duplicate the space?
*It could be great if with webui and maybe with the 40b version*
I'm looking to run these LLM on my local computer I don't like Colab -GPTQ vs Transformers models
It should work fine with the code in the Colab, if you have a GPU. Do you have a GPU (Nvidia) ?
you didn't use bitsandbytes, y did u install it?
If you did a full series on Falcon-7B and all the features of LangChain including memory and more advanced tools you will lead this space everywhere. All i see is OpenAI tutorials with LangChain and I am tired of paying $10 for every project.
Thanks for the suggestion, definitely worth going down that path!
Interested too
@@1littlecoder thanks I really really really appreciate it. To give you an idea I blew through $10 on a simple LangChain with OpenAI and it is simply not an option however every literal LangChain tutorial focuses solely on OpenAI and maybe a simple Hugging Face trivial example. The future is Falcon and Falcon-like models and having a comprehensive tutorial with Falcon and LangChain means that any model can be easily substituted. Excited if you do such a full series!!!
@@mr.yayaman yea this will be so helpful!
That would be amezing....
Thanks
can i run falcon 7b on uhd 730 ? 16 gb of cpu and gpu shared ram ??
Thanks a lot, can we do fine-tuning and indexing with falcon and lang chain combination. If possible, can you do a video on that
I remember seeing some fine-tuned falcon version, I've not tried myself yet
@@1littlecoder thanks for replying back, more power to you ❤️
Great video as usual
Thanks :)
How to speed up prompting and deploy this model for an app? Problem is in my IDE, because of my low GPU probably, it gives me an answer in like 2 hours xd
Is it like a CPU you've got ?
Can we use Conversational Retrieval Chains with this LLM to chat with PDF in langchain ?
I haven't tried it yet
Hi , I am getting this error - ImportError Traceback (most recent call last)
in
----> 1 from langchain import HuggingFacePipeline
2 from transformers import AutoTokenizer, pipeline
3 import torch
ImportError: cannot import name 'HuggingFacePipeline' from 'langchain' (/opt/conda/lib/python3.7/site-packages/langchain/__init__.py) when i run it on Sagemaker notebooks. Please help. I am not using colab.
GRacias!!!
Great job finally a LangChain with Falcon rather than the paid OpenAI bravo!!!
I'm still not sure how good this is, but the point of releasing this for the community to explore and share more :)
@@1littlecoder I have not seen anything like it so far. I think there is tremendous value here. I would go deep in all aspects of LangChain with Falcon.
How to upload pdf and ask questions
It is very slow, making it run on the PC, compared to llama, or alpaca turbo, or gpt4all, the truth is, I expected it to have better performance, and I can even say that the answers it gives were not at all up to those mentioned previously,
I need to try it more, another thing that I noticed is that I was using the 12gb of video-ram of the rtx, and the processor used 100% but I can say that it took me "minutes" to answer some things and the answers were very bad. that with the model 7b
Good to know
Hello! thanks for the video.
How can I manage model storage when running code on Windows using VS Code for use in a Docker container? Also, how do I ensure the model is only downloaded once, as it currently only works correctly on Colab?
machine!
how can we fine-tune this?
Video coming soon
That colab is a lie, I changed the propt to generate another response and not only does it take longer but it tells me below the response of the AI that was included, that is, somewhat misleading to appear faster
You called it a lie because something didn't work as it's supposed to
Can you make a video on the best open source commercial allowed llms?
Falcon scores the top of LLM leaderboard and is commercially allowed. Perhaps, I can make a listicle!
Bro it would be grateful if you do it with flowise
Thanks Bro. Flowwise is in my list of things to do unfortunately list of things are just going up :)