GPU VRAM Calculation for LLM Inference and Training

Поделиться
HTML-код
  • Опубликовано: 11 сен 2024

Комментарии • 8

  • @josephj1643
    @josephj1643 Месяц назад

    Exactly what I was expecting!

  • @LandAhoy-dx9nw
    @LandAhoy-dx9nw Месяц назад

    Great video.. Got a question. Is the batch size = number of concurrent users or concurrent queries? Thanks for any pointers on advance!

  • @CryptoMaN_Rahul
    @CryptoMaN_Rahul Месяц назад

    Hi bhaiya!!
    I'm working on my final year project, basically it has 2 ideas .
    1) AI POWERED previous year paper analysis system and sample paper generation from the current trends
    2) AI powered notes generation from the textbook content.
    IN TOTAL THERE ARE 7 ENGINEERING DEPARTMENTS IN OUR COLLEGE
    I'm little bit confused what to use where , agentic RAG , fine tuning or any other things ??
    Please help me to clear my confusion
    Thanks!!

  • @ohochman
    @ohochman 20 дней назад

    But how do I know how much time it will take to train my LLM model on A100 - is 5-days processing on GPU is good ? bad? it there any way to estimate that time of processing?

  • @muhammedaslama9908
    @muhammedaslama9908 Месяц назад

    Amazing

  • @user-cb7yl4nr6h
    @user-cb7yl4nr6h 22 дня назад

    How do I run and train a model on an integrated VGA uhd770 on i714700intel?

  • @andres.yodars
    @andres.yodars Месяц назад

    thanks

  • @akiivoice
    @akiivoice Месяц назад

    Ek number bhai par smj na aaya kuch bhi 😮