How to Use Meta Llama 3 with Hugging Face on Google Colab: Easy Step-by-Step Guide

In this video, I'll show you how to set up and use the Meta Llama 3 model with Hugging Face in a Google Colab notebook. We'll go through each step in detail, from installing necessary libraries to generating text with ease. Whether you're a beginner or an experienced developer, this guide will help you harness the power of Meta Llama 3 efficiently. Don't forget to like, subscribe, and hit the notification bell for more AI and tech tutorials!

[@hasnainazam3746]

Could you please share the colab link ?

[@RACM27MD]

A few tips to run this as 5thf of August 2024 with Llama 3.1 8B Instruct:
Next to pip install transformers add upgrade transformers:
```
!pip install transformers torch accelerate bitsandbytes
!pip install --upgrade transformers
```
This is the full import section:
```
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig, AutoConfig, pipeline
from huggingface_hub import login
```
Hugging Face Login, modelid and config:
```
login(token=hf_token)
model_id = 'meta-llama/Meta-Llama-3.1-8B-Instruct'
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type='nf4',
bnb_4bit_compute_dtype=torch.bfloat16
)
tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.pad_token = tokenizer.eos_token
config = AutoConfig.from_pretrained(model_id)
config.rope_scaling = { "type": "linear", "factor": 8.0 } # Adjust the factor as needed
model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config, device_map='auto')
```
Text generator:
```
text_generator = pipeline(
'text-generation',
model=model,
tokenizer=tokenizer,
max_new_tokens=512,
)
```
Everything else can stay the same
Also, go to Runtime -> Change runtime type and select the GPU option.
And don't forget to ask for access to Llama on hugging face. It won't work if you're not approved.

[@TheOpenSourceChannel]

Thank you for sharing!

[@EgidijaM.]

@RACM27MD - you saved me! Thanks a lot for such helpful notes!

[@tki967]

my_secret_key has never been used?

[@TheOpenSourceChannel]

Hi @tki967, Actually I have set the HF_TOKEN as an environment variable(by adding secrets in colab notebook). Hugging Face libraries can automatically pick it up without needing to pass it explicitly in the code. So you don't need to pass the token in your code.
But if you want to, you can pass it directly in tokenizer as below:
tokenizer = AutoTokenizer.from_pretrained(model_id, token=my_secret_key)
I forgot to explain this in the video. Sorry for the confusion.