EASIEST Way to Fine-Tune LLAMA-3.2 and Run it in Ollama

Поделиться
HTML-код
  • Опубликовано: 13 янв 2025

Комментарии • 51

  • @kunalr_ai
    @kunalr_ai 3 месяца назад +14

    Here are the key points from the video:
    * Meta released a new family of four different models, including multimodal models, called LLaMA 3.2.
    * The models are impressive for both language and vision tasks for their respective sizes.
    * You can fine-tune LLaMA 3.2 for your own custom tasks.
    * You can use Unslot for fine-tuning and Ollama for running the fine-tuned model locally.
    * The 1 and 3 billion models are particularly interesting because you can run them on device.
    * Meta has also released LLaMA Stack, which is their opinionated version of how developer experience should look.
    * You can fine-tune one of the smaller models on your own data set and then run it locally using Ollama.
    * You will need to provide your own data set and follow the specific prompt template used by the model.
    * You can use the official notebook from the Unslot team to fine-tune LLaMA 3.2.
    * You can use the supervised fine tuning trainer from the TRL library to train the model.
    * You can save the trained model as a GGf file and then load it in Ollama.
    * You can create a model file in Ollama and then run the model using the AMA run command.
    Timeline with tags:
    00:00 - 00:15: Introduction
    00:15 - 02:00: Meta releases LLaMA 3.2
    02:00 - 04:00: LLaMA 3.2 models
    04:00 - 06:00: Fine-tuning LLaMA 3.2
    06:00 - 08:00: Unslot and Ollama
    08:00 - 10:00: 1 and 3 billion models
    10:00 - 12:00: LLaMA Stack
    12:00 - 14:00: Fine-tuning LLaMA 3.2 on your own data set
    14:00 - 16:00: Prompt template
    16:00 - 18:00: Unslot notebook
    18:00 - 20:00: Supervised fine tuning trainer
    20:00 - 22:00: Saving the trained model
    22:00 - 24:00: Running the model in Ollama

  • @gramnegrod
    @gramnegrod 3 месяца назад +2

    Great video. You make it look so easy! I’m really looking forward to the vision based rag. I’m hoping good vision models with vision rag will open up a lot of creative use cases.

    • @engineerprompt
      @engineerprompt  3 месяца назад +1

      Here are a couple of examples of vision based RAG:
      ruclips.net/video/w5WGbUGAE3s/видео.html
      ruclips.net/video/DI9Q60T_054/видео.html

  • @epokaixyz
    @epokaixyz 3 месяца назад +16

    Consider this your cheat sheet for applying the video's advice:
    1. Research the different sizes of Llama 3.2 models.
    2. Download the Unsloth Fine-Tuning Notebook.
    3. Acquire the FineTome-100k dataset.
    4. Fine-tune a Llama 3.2 model with Unsloth, using LoRA adapters and prompt engineering.
    5. Create an Ollama model file for your fine-tuned model.
    6. Run your fine-tuned Llama 3.2 model locally with Ollama.
    7. Start building custom AI applications!

    • @Criminal_H4_ff
      @Criminal_H4_ff 3 месяца назад

      Broo till step 6 i have completed but i cant run my finetunned model in ollama what should i do now 🤧

  • @lulzkiller666
    @lulzkiller666 3 месяца назад +8

    Nice video. Could you please make a video on how to train it on "own" content. Lets say, i have the complete API documentation for an APP, i want to train it on this API documentation so that it can help me code faster with the correct API's. That would be awesome

  • @AshwaniKumar-r4p
    @AshwaniKumar-r4p 2 месяца назад

    In the fine-tuning process demonstrated in the video, does the model primarily learn response patterns, or does it genuinely absorb and retain the specific knowledge contained in the training dataset?

  • @deschwedda
    @deschwedda 3 месяца назад +11

    I want to exceed limitations and remove censorships. Is it possible, and how? thank you so much.

    • @Incredible_428
      @Incredible_428 3 месяца назад +3

      You need to fine-tune it with a dataset which contains uncensored chat data, it should be well mannered/structured so that the model will learn batter patterns

    • @deschwedda
      @deschwedda 3 месяца назад +1

      @@Incredible_428 thank you, any dataset recommendations? (llama 3.2)

    • @engineerprompt
      @engineerprompt  3 месяца назад +6

      look for dolphin models, they are usually uncensored.

    • @deschwedda
      @deschwedda 3 месяца назад

      @@engineerprompt thank you!

    • @tnix80
      @tnix80 3 месяца назад +1

      If you can jailbreak AI and the woke nonsense, a lot of people are going to want to use your jailbreaking technique/tool. I could see making a lot of money.

  • @rezcan
    @rezcan 2 месяца назад

    Great video, thanks can you make a video to show how to fine tune Llama 3.2 90B vision model?

  • @dipeshrathore8842
    @dipeshrathore8842 3 месяца назад +4

    Great video!
    Can you please create a video or guide demonstrating Fine-tuning of Llama 3.1 8B
    First on raw text (books, discourses etc.)
    Then on instruction dataset (less data 8-10k)?
    And what's best? 8B-base or 8B-instruct for this?! (I don't wanna lose general chat capabilities)

  • @maxwelikow9119
    @maxwelikow9119 Месяц назад

    Cool video! Do you have an idea how to fine-tune the llama3.2-vision models?

  • @Abhijit_SU
    @Abhijit_SU 3 месяца назад +2

    I have used the same notebook to fine tune my model.
    I am getting an error saying "Keyerror: name" when i am trying to either push it to HF or saving it locally.
    After executing the GGUF / llama.cpp Conversion part it is running and then after 3 mins exact it is showing the error every time. Please tell me how did you manage to download the GGUF file locally using the same Notebook which You have provided. Please Help, Thanks In Advance !

  • @raunaksharma8638
    @raunaksharma8638 3 месяца назад +1

    Can we use normal Alpaca type dataset with input , output and instruction here ?

  • @tanguyvansnick3954
    @tanguyvansnick3954 2 месяца назад

    Great tutorial 🔥

  • @tapikoBlends
    @tapikoBlends 2 месяца назад

    is it possible to make fine-tuning using text?(not structured in json format)
    text will be tomething like instruction

  • @HarshSingh-cp8mq
    @HarshSingh-cp8mq 2 месяца назад +1

    hey i want to build my personal assistant on the LLAMA3.2 and i want to assign a name to it. Also while asking the owner it tells me about meta this also i want to change?? Can anybody guide me

  • @surajsingh-iw8yt
    @surajsingh-iw8yt 29 дней назад

    wht would be the format of custon tuning dataset, like file format jsonl or someting else

  • @annwang5530
    @annwang5530 3 месяца назад +1

    Can that gguf run locally on DAN or LMStudio?

    • @xmagcx1
      @xmagcx1 3 месяца назад

      x2

    • @engineerprompt
      @engineerprompt  3 месяца назад

      Yup, on almost anything you want, if its based on llamacpp.

    • @annwang5530
      @annwang5530 3 месяца назад

      @@engineerprompt do you take fine tune tasks? I got a Json dataset I fail to fine tune...

  • @Qwme5
    @Qwme5 3 месяца назад

    Can I finetune this llm with a new langauage like Arabic if so should I use the original tokenizer of llama 3.2.
    Another question , how much it will cost me on google colab to finetune such small model like 3B.

    • @avataraang3334
      @avataraang3334 3 месяца назад +1

      Nothing.. T4 gpu gives you around 1-3.5 hours of free resource.. thats plenty so smaller models

  • @lysanderAI
    @lysanderAI 2 месяца назад

    Could you do a video of finetuning using axolotl + unsloth

  • @lewists9475
    @lewists9475 24 дня назад

    can a llama 3.2 vision be finetuned and run in ollama?

  • @MuhammadAsif-mm4py
    @MuhammadAsif-mm4py 3 месяца назад

    Can i use this model in my android application? Please help

  • @yan_yan_1995
    @yan_yan_1995 3 месяца назад

    may i know the screen recording software he's using ? it's cute !

  • @sergiosilveramurcia4216
    @sergiosilveramurcia4216 3 месяца назад

    Has anyone tried to run it locally on MacOs, does it change the code substantially?

  • @aqibjamal1077
    @aqibjamal1077 Месяц назад

    make video for 11b vision model please

  • @nazarmohammed5681
    @nazarmohammed5681 4 часа назад

    Rather than fine tuning on hugging face data set. Plz show fine tuning on own data set

  • @jackluka773
    @jackluka773 3 месяца назад +2

    why the heck it has to be so complicated? can't it be wrapped in some easy to use GUI with drop down list creator with description of the consequences for each choice?

    • @yufeixu4479
      @yufeixu4479 Месяц назад +1

      You go program it then 😂 this is easy already... if you can't do it then don't do it and don't hate buddy

    • @equious8413
      @equious8413 Месяц назад

      This is cutting edge computing science. Paint by numbers aint here yet.

  • @大支爺
    @大支爺 3 месяца назад

    Uncensored patch first!

  • @justinln6019
    @justinln6019 Месяц назад

    Why do I get this error:
    RuntimeError: Unsloth: The file 'llama.cpp/llama-quantize' or 'llama.cpp/quantize' does not exist.

  • @robertjalanda
    @robertjalanda 3 месяца назад

    great video waiting for vision support

  • @Criminal_H4_ff
    @Criminal_H4_ff 3 месяца назад

    Brother i got error while doing the command ollama run mymodelname it throws the error as ollama runner function terminated and vocabulary and tokenizer merges files are not found issue what should i do now will you please any contact of yours i need immediate help bruh😮‍💨🥲