Видео

Thanks for leaving all the errors in and correcting them. Excellent.

tryin, 10 turing pi's with 40 nvidia jetson orin's on them, with primary linux computer, so 41 systems total, clustered using archer and kubermites. it will def rocket then... 4,000-6,000 tops, 1.2 terabytes of ram.

How do you connect to server via Python Client or Fast APIs for integration with projects/notebook?

TG largely depends on memory-bandwidth (the SoC has to pump all of the parameters and the KV-caches from RAM into the SoC's caches for each token generated). PP (and ML) is dependent on compute (GPU-horsepower) because token-processing can be batched. The M4 has 20% faster memory-bandwidth in addition to the faster GPUs. Let's see when Apple will do MacBooks with these chips, maybe I will upgrade my M2. For me, the M3 is not interesting enough for an M2 upgrade.

Please stop click baiting

i cant run it on service ollama start, it says the following: $sudo: service ollama start ollama: unrecognized service

Sir I'm off topic but how can I enable usb support on the steam deck Gnome Boxes

Hi Ian, great clip - how do we get it to read the prompt answers aloud with reasonable low latency?

Great! But try Gemma model. Sounds strange, but it is really good on picking content.

Hey Ian, there is a more Ollama native library for the same use case called ollama-instructor. It was inspired by instructor from Jason Liu.

thank you.

What about renew this video with the new Rpi Hat AI? Thanks

Great stuff, thanks a lot!

hello Ian, Its a very great video. I have some query, i will very thankful if you can help me. I am stuck since 3 days. Apparently, I am trying to host the ollama on my server. i am very new to linux and dont understand the whats wrong i am doing. I am using nginx to host the ollama on my proxies and configure the nginx file and yet getting access denied error. I can show you the code if you want, please respond.

I had no idea brew was also available for linux. Pretty cool way of going about it

I want to start experimenting with llms and I have a budget for laptop or pc or a compromise of both , I was going for a great Mac or an ok one and a pc , what’s your advise ?

That's sick!!!

Very helpful thank you so much

Says error when pulling list

How much was the new compared to refurbished?

thank you🦙

Traceback (most recent call last): File "/Users/*******/Desktop/facefusion/run.py", line 3, in <module> from facefusion import core File "/Users/*******/Desktop/facefusion/facefusion/core.py", line 9, in <module> import numpy ModuleNotFoundError: No module named 'numpy'

Hi Ian, thank you for the video! When I issue the run command, I'm getting a "Frame processor .DS_Store could not be loaded" error. Do you have any suggestions to fix this?

Not simple at all, you just go ahead with it assuming we already knew the basics

I think your solution using brew is better than using distrobox. For some weird reason (I couldn´t find out why) more than a dozen times I had the whole distrobox "broke" for any weird reason. I could not understand why it happens, but all of the sudden distrobox would not initiate the image. Something like podman had lost track of where the volume of the image was. So for podman the container was still running. And it was indeed running an external container (which was the volume). This have happened to me at least 7 times. Dunno if it is when you switch to gaming mode some bad-process management happen and the process of podman gets lost -- or some file gets losts in some process of the updates. I've given up trying to figure out why this kept on happening. And once it does, it is almost impossible to retrieve the volume of your old container, so it is the same as starting over. So .. imo it is way easier and better setup to use brew. But if you installed distrobox and did not find any error, just go along! prob was on my deck only.

I've trie drunning OLLAMA on my Raspberry Pi 5, as well as an Intel Celeron based computer, and also an old Intel i7 based computer, and it worked everywhere. It is really behind impressive, thank you for this video to show me how to do it!

Does it automaticall detect and use apple m2 gpu? is there anything I need to configure to use it with gpu?

Cool

i know this video is old but can you or someone make a video on the white dev board with the pin connections and how to use it?

is there 2and 3 key versions of this board?

Does it run on CPU or GPU ? Thx

I just read your article on running the brew package manager on the Deck!

Thank you for this! The two main things that I dislike about LLMs is the middle school level answers and the nanny rails. Hopefully, running an uncensored LLM will at least make the low intelligence level less grating.

Run Pod is very affordable too. From 17c per hour for a Nvidea 3080

2:54 love the honesty of hitting the bug and keeping it in the video ❤

that's super impressive even if it takes quite a while to respond, is a RASPBERRY PI

Thanks for video and testing. I was wondering if you have tried setting num_threads =3. I can't find video of where I saw this but I think they set it before calling ollama. Like environment variable. It's supposed to run faster. I'm just building a rpi5 test station now

Thannks Ian! I really wanted to try doing some simple stuff with my steam deck on the go and this is just the way

What terminal alternatives do you recommend?

If they built up an ARM based Framework Laptop, I would buy that in a heartbeat.

Am I the only one who noticed tiny llama response to “why sky is blue?” was shitty? What the heck rust causing blue color of the sky?

That's reassuring...you're supposed to be the expert teaching the public, and you don't even cover all the standard classes in the module because you admit right off the bat that you 'only have experience' with a some?? I'll be sure to watch this until the end...or not.

Poetry is extremely slow when resolving the dependencies. And I m stucking this matter. poetry add hopsworks that it took for a long time. And finally it' not working. It's worked to forever

I just got a RPi 5 and ran the new Llama 3 (ollama run llama3). I was not expecting it to be this fast for something that is on the level of GPT-3.5 (or above). On a Raspberry Pi. Wow.

shockingly simple? you just highlighted how technical skills are needed and it's not for beginners lol that tutorial was awful

Very helpful thanks!

Do you know how to get it to run in LangChain while taking advantage of the M1/2 chips?

Such a calm tutorial but so informative💙

Awesome! I want to try this now! Can someone tell me if it necessary to install the model on an exterior SSD?

I didnt think you could use a zero 2w in the original case