Step-by-step guide on how to setup and run Llama-2 model locally
HTML-код
- Опубликовано: 8 сен 2024
- In this video we look at how to run Llama-2-7b model through hugginface and other nuances around it:
1. Getting Access to Llama Model via Meta and Hugging Face:
Learn how to obtain access to the Llama language model through Meta and Hugging Face platforms.
2. Downloading and Running Llama-2-7b Locally:
Follow step-by-step instructions on downloading the llama-2-7b model and running it on your local machine.
3. Tokenizing and Inputting Sentences:
Understand the process of tokenizing and inputting sentences for next-word prediction tasks using the Llama model.
4. Controlling Temperature Parameter:
Explore techniques for adjusting the temperature parameter to influence the creativity of Llama's output.
5. Challenges in the Base LLM Model:
Identify and address potential challenges and limitations associated with the base Llama language model and why one would go for fine-tuned model.
6. Choosing the Best Performing LLM:
Stay informed on how to check for the latest and best-performing Llama language models, ensuring optimal results for your tasks.
References and Links:
Previous Video on LLM concepts: • A basic introduction t...
Code: github.com/opp...
Llama 2 paper: arxiv.org/pdf/...
Huggingface: huggingface.co
Open LLM Leaderboard: huggingface.co...
Linkedin: / yash-agrawal-a22597162
Great video, and it is indeed the right translation to french :)
Thank you for the very nice presentation and explanation! I would like to a video with your wonderful explanation where you can tell us how we can fine-tune the base models to one fitting our specific tasks
Noted, will do it. Thanks!
Thanks! this was my first AI development video watch.
24:39 - Fun part of the video, good luck Yash ! Thanks for the video.
Great video. Thank you ycopie!
Very helpful. Thanks a lot for this.
Thank you for this hands-on! Initially I tried it on my laptop which although has an NVDA GEFORCE GTX, it can't run very well. Eventually I have to run it on Colab (T4 GPU), though not with adding the following lines to help with the GPU usage (just sharing) :
!pip install accelerate
from accelerate import Accelerator
accelerator = Accelerator()
device = accelerator.device
my model needs 16.2gb of GPU, which in colab is limited to 15gb, do you have any way to fix that?
really subtle! subscribed
Thanks for this discussion !
Thank you so much for your time!!!
Please share you system specs, specially about GPU you are using
i'm new to LLM and i just wanted to know that you need all these access for using llama, but when you'd use ollama you just put "ollama run llama2" in the terminal, so whats the difference? they can access it without any explicit access from meta??
Really great explanation.....
But i am stuck in a problem of getting space on gpu. If i tried this on google collab ,the free version gets collapsed due to all memory usage. Pls suggest me for this solution or list the name of small models that are under 12gb of space & are used for prompting purpose.
Thanks for your time... Please may I ask how to download coda toolkit on my laptop to support GPU support. The code for Coda or cpu is not working on my laptop
Do you offer any paid consulting? I’m stuck on an installation error.
Thank you so much for this video.
Could you please let us know how to connect with SQL database to fetch the information and implement semantic analysis?
I currently have the problem that it only says "Loading widget..." when I try to run the code and doesn't display the progress bar. Do you possibly know how to fix this?
good job!
nice tutorial but how would you do to wait for the prompt.. so we can enter the prompt like what is capital in Indisa and press enter.. then the model should reply.. how to do it.
what will we do if i need interactive mode, like having conversation like we do with chatgpt
how to fine tune with our own data sets, like answer the pdf of our own data sets.
good vid
can i usethis in vscode?
love your username lol
ConnectionError: ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))
bro iam getting this error when running on jupyter notebook.
please help.
Great vid - Thanks a mil! I am getting KeyError: 'llama' when running the script. I have copied in the model name/path from hugging face directly but its still causing an issue - Do you know what the problem could be?
Will need more details like which line in your code is causing this
@@ycopie1126 Sorry i think it actually was an issue with the version of transformers I had installed - i reinstalled and now the model seems to be downloading successfully - so all good! Appreciate your reply!
ValueError: You are trying to offload the whole model to the disk. Please use the `disk_offload` function instead.
i got this error
hf represents human feedback not hugging face
You can follow this discussion: github.com/facebookresearch/llama/issues/612
The model card has small difference which states that it's hugging face format.
Extremely sorry for doubting, just thought hf should mean human feedback, again I am wrong, sorry for that, good job
No worries at all. Happy you put it in comments so it would help other people as well 😄
Can you please tell me your PC specs ?
Hi when running this code I am getting an error File d:\Magister\llama_hugging_face\venv\lib\site-packages\huggingface_hub\utils\_validators.py:110, in validate_hf_hub_args.._inner_fn(*args, **kwargs)
109 if arg_name in ["repo_id", "from_id", "to_id"]:
--> 110 validate_repo_id(arg_value)
112 elif arg_name == "token" and arg_value is not None:
...
)
(norm): LlamaRMSNorm()
)
(lm_head): Linear(in_features=4096, out_features=32000, bias=False)
)'. Please provide either the path to a local folder or the repo_id of a model on the Hub.
Did you encounter this problem?
ConnectionError: ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))
bro iam getting this error when running on jupyter notebook.
please help.