Is it possible to run the tiny llama model without the open-webUI and docker? I want to do a tiny bit of reinforcement learning on the model and then put it in my pi, and integrate it onto my local website.
Sure, I've tried bigger models without the AI kit, but the bigger they are the slower they run. Jeff Geerling did a demo on his channel using the AI Kit for visual recognition stuff and it was pretty amazing. I also wonder how its performance on LLMs would improve.
Would I be able to connect an lcd screen, microphone module, speaker module, and etc. and run the llm as a handheld device? Also what changes in the code would it require?
You could do this. There are modules for the raspberry pi that allow the connection of a monitor etc. In my project, I mainly used the pi as a server and accessed the LLM through a browser on a mobile phone connected to the pi's WiFi.
I would love to try that out someday. I currently don't have that stuff though. Saw some videos on Jeff Geerling's channel that were very interesting though.
Deploying an LLM to a cloud is easy using the docker-compose.yaml file in the Pillama repository, though getting it to work well with the cloud's infrastructure will be very case-by-case depending on how your particular cloud is setup. Using cloud-based GPUs etc will yield better results, so I'd suggest looking through the documentation for your cloud infrastructure provider. Sorry, I don't have a one-size-fits-all answer for this question.
It depends on what your goal is. If you want a blazing fast AI, yeah it's not worth it. On the other hand, if you want to learn more about how these things work and perhaps how you can serve your own AI later on more impressive hardware (or on platforms like AWS and/or Azure) then it's totally worth it. Or you could do it to win a beer bet 😉 In our case, we did it simply to show it CAN be done. To demonstrate to people that AI is not beyond them and to thus empower people. This could be the start of a journey for people who otherwise might be too self-doubting to take their first step. That makes it worth it to me. Oh, and it's cute and fun too...
![Stromae, Pomme - “Ma Meilleure Ennemie” (from Arcane Season 2) [Official Visualizer]](http://i.ytimg.com/vi/1F3OGIFnW1k/mqdefault.jpg)
Great video 👌 full of useful information.. thanks 🙏
Glad it was helpful! Thank you
Is it possible to run the tiny llama model without the open-webUI and docker? I want to do a tiny bit of reinforcement learning on the model and then put it in my pi, and integrate it onto my local website.
Great video, lots of fun here !
Do you think the Pi AI kit could run a bigger model ?
Sure, I've tried bigger models without the AI kit, but the bigger they are the slower they run. Jeff Geerling did a demo on his channel using the AI Kit for visual recognition stuff and it was pretty amazing. I also wonder how its performance on LLMs would improve.
Would I be able to connect an lcd screen, microphone module, speaker module, and etc. and run the llm as a handheld device?
Also what changes in the code would it require?
You could do this. There are modules for the raspberry pi that allow the connection of a monitor etc. In my project, I mainly used the pi as a server and accessed the LLM through a browser on a mobile phone connected to the pi's WiFi.
I would like to run this in docker swarm as a service on rpi's, any help there?
tinyllama and Coral or Hat AI (Hailo 8L)??
I would love to try that out someday. I currently don't have that stuff though. Saw some videos on Jeff Geerling's channel that were very interesting though.
Thanks
i have created my own llm so how i can deploy it on google cloud and use it on raspberry pi plzz tell me
Deploying an LLM to a cloud is easy using the docker-compose.yaml file in the Pillama repository, though getting it to work well with the cloud's infrastructure will be very case-by-case depending on how your particular cloud is setup. Using cloud-based GPUs etc will yield better results, so I'd suggest looking through the documentation for your cloud infrastructure provider. Sorry, I don't have a one-size-fits-all answer for this question.
That is so painfully slow, doesnt look worth it
It depends on what your goal is. If you want a blazing fast AI, yeah it's not worth it. On the other hand, if you want to learn more about how these things work and perhaps how you can serve your own AI later on more impressive hardware (or on platforms like AWS and/or Azure) then it's totally worth it. Or you could do it to win a beer bet 😉
In our case, we did it simply to show it CAN be done. To demonstrate to people that AI is not beyond them and to thus empower people. This could be the start of a journey for people who otherwise might be too self-doubting to take their first step. That makes it worth it to me. Oh, and it's cute and fun too...