What is Post Training Quantization - GGUF, AWQ, GPTQ - LLM Concepts ( EP - 4 )

Поделиться
HTML-код
  • Опубликовано: 27 окт 2024

Комментарии • 9

  • @har111100
    @har111100 2 месяца назад

    awesome playlist. Keep posting.

  • @sqlsql4304
    @sqlsql4304 6 месяцев назад

    well expalined.. Lot many concepts got cleared

  • @mitejmadan8672
    @mitejmadan8672 6 месяцев назад

    Hey akhil, could you make this course in such a way that after doing this one can at least apply at your company for an internship.
    If not then atleast make a roadmap mentioning all the keywords one can search and learn from the internet. Since i am a full stack I don't have much idea of the ai landscape.

  • @theuniversityofthemind6347
    @theuniversityofthemind6347 14 дней назад

    Hi Akhi, Hoping that you can help, I have an Alienware m18 R2 with an Intel i9-14900HX, NVIDIA RTX 4090 (24GB), 64GB RAM, and 8TB storage. For extra information I don't plan to use this for high intensive tasks like model training or any other such high intensive computing tasks, i only mainly will be using it for analysing my business documents and also writing 20 minute elaborate stories based on a five step story structure. I wanted to use the 70B model to generate the best possible results for these smaller less intensive type tasks. Based on my system specs which which optimisation method would you recommend. GPT Q, GGUF, or AWQ ad would you have any additional advice on the best way to optimise based on my use case requirements?

    • @AkhilSharmaTech
      @AkhilSharmaTech  13 дней назад

      @@theuniversityofthemind6347 hey bro I don't recommend doing anything on personal PC until you know what you're doing.
      Try doing everything on an AWS instance first

    • @theuniversityofthemind6347
      @theuniversityofthemind6347 13 дней назад

      @@AkhilSharmaTech Thanks for the reply Akhi. Thats understandable. Ok, do you at least think my system would be powerful enough to run a LLama 70B model for the described tasks?

    • @AkhilSharmaTech
      @AkhilSharmaTech  13 дней назад

      @@theuniversityofthemind6347 even though the system is quite powerful, I'd still suggest just using a 7B model max on your local system

  • @rahuldebdas2374
    @rahuldebdas2374 4 месяца назад

    very fast explanation. pls be slow from next time. hard to follow.

    • @AkhilSharmaTech
      @AkhilSharmaTech  4 месяца назад +1

      will do. also, try slowing down the video to 0.5x if possible.