Ollama: Run LLMs Locally On Your Computer (Fast and Easy)

Поделиться
HTML-код
  • Опубликовано: 25 июн 2024
  • With Ollama, you can run local, open-source LLMs on your own computer easily and for free. This tutorial walks through how to install and use Ollama, how to access it via a local REST API, and how to use it in a Python app (using a client library like Langchain).
    👉 Links
    🔗 Ollama GitHub: github.com/ollama
    🔗 LLM Library: ollama.com/library
    🔗 RAG + Langchain Python Project: • RAG + Langchain Python...
    📚 Chapters
    00:00 How To Run LLMs Locally
    01:07 Install Ollama
    02:45 Ollama Server and API
    04:15 Using Ollama Via Langchain

Комментарии • 31

  • @iamfine4891
    @iamfine4891 4 дня назад

    Love this channel, your content is clear explanation. Please do one video for fine tuning LLM for any specific task with one real-time use case

  • @kacperwodarczyk9349
    @kacperwodarczyk9349 2 месяца назад +1

    Love this channel, very clear and factual explanation of the topic

    • @pixegami
      @pixegami  2 месяца назад

      Thank you, I really appreciate hearing that :)

  • @mayankbhadauria1
    @mayankbhadauria1 Месяц назад +1

    First I would like to thank you, As a beginner you have provided a solid platform to me. I would also eagerly wait for your next video on how to train model locally and fine tune . Thank you so much once again .

    • @pixegami
      @pixegami  Месяц назад

      Thank you, glad you liked it!

  • @matheussimonacivieira9487
    @matheussimonacivieira9487 Месяц назад +1

    Thanks, the simple codes you showed helped me a lot!

    • @pixegami
      @pixegami  29 дней назад

      Glad to hear that!

  • @malkitsingh01
    @malkitsingh01 2 месяца назад +1

    Sound quality is much better in this new setup. I just saw your FastApi video (if you are wondering why I am appreciating the sound of the video 😂)

    • @pixegami
      @pixegami  2 месяца назад

      Haha I'm glad to hear that. I made some adjustments to my set-up, glad it's paying off :)

  • @mohamedjamaludeen7179
    @mohamedjamaludeen7179 Месяц назад

    It's Good. Very nice explanation😀

  • @mzimmerman1988
    @mzimmerman1988 2 месяца назад +1

    thanks

    • @pixegami
      @pixegami  2 месяца назад

      You're welcome!

  • @lesptitsoiseaux
    @lesptitsoiseaux 26 дней назад +1

    Diving a bit deeper into embeddings would be nice. And vector database. How to know the quality of your embeddings. What made you go to bedrock?

    • @pixegami
      @pixegami  19 дней назад +1

      Ultimately, the only way to test your app effectively is to do end-to-end testing with a bunch of sample answers/questions. If you get good results, and the embeddings help you find the right items in the DB, your embeddings are probably good enough.
      As for Bedrock: my decision wasn't exactly to use Bedrock-it was more to use a larger cloud-based model (OpenAI or Gemini would be fine too). I just used Bedrock because my developer stack is quite biased/skewed towards AWS, so I guess just personal choice.

    • @lesptitsoiseaux
      @lesptitsoiseaux 18 дней назад

      @@pixegami Could I hire you to assist me on a project? Very similar to your tutorial, it'd save me time?

  • @alexandrosanapolitanos-ew4ox
    @alexandrosanapolitanos-ew4ox Месяц назад

    Super interesting! Do you know what are the RAM requirements to run this locally?

    • @manoharmeka999
      @manoharmeka999 28 дней назад +1

      How you must frame your question is how the system requirements are changing from OLLAMA 1 to OLLAMA 3? So even if you invest heavily now, when the future versions come out and as LLMs keep growing size exponentially, there's no point running locally unless you're investing in hardware that requires to be upgraded every 4 years. Whatever you earn as bonus every year, keep it aside to invest in hardware, online tutorials and books.
      Finally you could put up your question to ChatGPT itself instead of seeking answers here.

    • @alexandrosanapolitanos-ew4ox
      @alexandrosanapolitanos-ew4ox 25 дней назад

      @@manoharmeka999 you seem to be very strongly opinionated. But there are applications like when you are dealing with sensitive documents where you might not want to expose this info to open ai or anyone else via a query. Also that money might be a lot for people in India but are just a business expense for some others.

  • @Screonizma
    @Screonizma 2 месяца назад +1

    It's interesting to train custom LLM instead of using RAG [2:45]

    • @pixegami
      @pixegami  2 месяца назад +1

      I haven't looked into training an LLM. It's a bit more challenging and expensive to do than just using an off-the-shelf model, but it's a great way to gain more control and quality from the LLM.

    • @Screonizma
      @Screonizma 2 месяца назад

      @@pixegami there is the limit for llm context so it's hard to convert knowledge base into a little text. There are no lot of examples of using ldap3 library on python, even llama3 knows it. But it's hard to produce example with my own messaging library in my corporation.

  • @Thelgren00
    @Thelgren00 21 день назад +1

    Can i install ai town with this. Other method was too complex for me as i am new to alot of this

    • @pixegami
      @pixegami  19 дней назад +1

      Sorry, I'm not familiar with "AI Town" - is it this? github.com/a16z-infra/ai-town
      It looks like you can use Ollama as a backend: github.com/a16z-infra/ai-town?tab=readme-ov-file#3-to-run-a-local-llm-download-and-run-ollama

  • @jimmylin8538
    @jimmylin8538 2 месяца назад +1

    can you do new episode combine with "Ollama: Run LLMs Locally On Your Computer (Fast and Easy)" and "Langchain Python Project: Easy AI/Chat For Your Docs"? Which you will just used local LLM to process the Docs

    • @pixegami
      @pixegami  2 месяца назад

      Absolutely :) A lot of people have been asking for this, so that's going to be my next video (plus a couple of other top requested features).

  • @arwazg6150
    @arwazg6150 Месяц назад +1

    can i integrate ollama with java?

    • @pixegami
      @pixegami  Месяц назад +1

      Doesn't seem like there's a first-party Java integration, but there are some third party ones: github.com/amithkoujalgi/ollama4j
      Or you can use Java to make a standard REST API call to the local server directly.

  • @morespinach9832
    @morespinach9832 2 месяца назад +1

    How about Meta’s LLM?

    • @pixegami
      @pixegami  2 месяца назад

      Absolutely. This is available on Ollama. You can use `llama2`, but now `llama3` is also available: ollama.com/blog/llama3