Is the new Raspberry Pi AI Kit better than Google Coral?
HTML-код
- Опубликовано: 20 авг 2024
- I've got my hands on the new Raspberry Pi AI Kit, an M.2 Hat with the Halio 8L module onboard. This brings advanced AI capabilities to the Raspberry Pi 5, and offloads this to the Neural Processing Unit.
Join this channel to get access to perks:
/ @kevinmcaleer28
💁♂️ For more information on SMARS Robots, tutorials and more visit: www.smarsfan.com
🎖To join the membership at 🥉bronze, 🥈silver or 🥇gold levels, head over to
www.smarsfan.c...
Join the list - action.smarsfa...
☕️ Enjoy this video? Buy me a coffee! www.buymeacoff...
📸 Follow me on Instagram - @kevinmcaleer / kevinmcaleer
🐦 Follow me on Twitter - @kevsmac / kevsmac
🙂📘 Join the Facebook group - Small Robots / smallrobots
👩💻 My Code on GitHub: www.github.com...
🎵 Music by Epidemic Sounds www.epidemicso...
#Pico #MicroPython #Robotics
Thanks for the video. I got one of these kits and it is amazing when running the provided sample codes/apps. Pity to program something leveraging on this kit you need c++, the installation of the Hailo SDK and then it becomes very complicated for 99% of the people using a Pi. I would not recommend the AI Hailo kit for everyone. It is for developers with good knowledge of AI and c++. There is not a single video explains that, all video on RUclips just try to market it, I have not found any tutorial explains how to modify or make a new app. At present, if you got one of these kits you will be on your own.
there is no other way around but learning before being capable of creating interesting stuff. There's a number of courses on C++ and ML on the internet
If you want to get something new, do something new, then you need to learn something new. Otherwise, just keep posting cat pictures on Instagram.
Google Coral has python support, which makes getting into using it considerably easier
@@seagsmtrashseagsmtrash1906 Thank you for your message. I appreciate the sentiment behind "if you want to get something new, do something new, then you need to learn something new." While I personally have no issues with C++ and other programming languages, I have some concerns regarding the advertisement of the AI kit for the Raspberry Pi 5 in a way that looks like it is easy for everyone.
All the videos I've checked on this AI kit don't mention the specific skills required to use it effectively. The Raspberry boards are predominantly used by hobbyists, and it's unreasonable to assume that all of them are proficient in C++. Therefore, it seems a bit too marketing-oriented to emphasize how easy it is to use the AI kit without clearly stating that a good understanding of C++ and the Hailo SDK is necessary.
Additionally, there isn't a single video available today showing how to use the Hailo SDK or providing tutorials on writing code that leverages this AI kit on a Raspberry Pi 5.
While some of the responses to my post seem to highlight "how good I am because I know C++," my intention was simply to direct potential buyers to investigate a bit further before purchasing this kit, which I consider amazing but.... again not for everyone.
True, I came to realize that luckily before buying. Unfortunately hailo doesn’t release the dataflow compiler, so it’s basically useless when trying to infer any „small-LLM“ like tinyllama or such.
Currently it seems that the cheapest and most energy efficient alternative for any practical assistant-like LLM is the mac-mini, but the price tag - oof.
Perhaps the orangepi would be a bit more accessible, but I have yet to see any LLM on it
A great option for robot vision, IMO. I still wish there were at least a little NPU right on the Pi (like the RK3588)... but this is a worthy option in lieu of that.
Also regarding the TOPS/W rating, I had 13 TOPS at 1.5W-according to their specs, but I was wondering if you are using the full Pi + Hailo-8L power consumption for your rating of 3 TOPS/W?
I’ll look into that shortly Jeff - what equipment do you use to test the power draw?
Ah, I may have been reading the specs wrong-realized they are using "1.5W" on their site to indicate typical usage at 30-60 fps of a standard image processing model. It can use more than that when it goes all-out at 13 TOPS. Wish I could edit my video ha!
@@kevinmcaleer28 I use a ThirdReality Smart Outlet for measurements over time, it's connected to my Home Assistant. I have also used a few other power measurement devices, but the ThirdReality one's been the most reliable and seems pretty accurate!
Agreed. I was waiting for a promising tensor accelerator for the pi for a while. I just bought this to try it.
A possible advantage is that you can use the ram on the pi, and offload computation via the PCIe3.0 accelerator. This could allow to use larger models, a problem I have with e.g. the oak d lite is the 2GB ram limitation.
Something I want to try is to run a local LLM on the pi using halio acceleration. This could give a low latency LLM for things like e.g. Speech to Text to Speech. it's something that works poorly on the cloud due to the latency involved.
I'm quite interested in seeing its performance running LLMs via Ollama. With a RPi5 alone, it takes vision model LlaVa about 5 whole minutes to interpret an image. I'm hoping the AI kit can improve that vastly! Have you given it a go?
Hiiii Kev! 👋you could make your fortune telling robot EXTRA spookily accurate with this kit.
Cool! Yes, if I added a camera and ollama back end it could detect a person and then say something spooky about what it sees!
It great to have such a chip design with tightly integrated memory which is obvious the way to go for any inference tasks. But ive found it interesting how hailo manages to keep any information about the amount of that super useful memory 100% secret; and this product launch is no exception. Like, dont we all want to know what models we can actually run on there? Arnt we going to find out eventually what the memory limits are after we spend our money? Or is that somehow the point?
Very interesting. It would be nice to hear more about supported frameworks in future. Realize this a hardware unboxing and quick demo.
Yes I’ll be following up with more info on that now that the full software stack is released
Hey I got one of these but was disappointed you can't use it with PyTorch or llm models or other models without some weird recompiling but they don't provide the full sdk. So if you wanna train something to find a squirrel for example, I don't see any way. In the mean time it's just a simple toy to run some of their demos. Unimpressive imo. I think the coral is better since it's easier to get it to help with tasks..
Extremely helpful video for beginners
I can imagine nature watchers will want this to identify animals
Very informative, I was looking for an accelerator for mobile robotics and found nothing useful. The Orin is powerful but has nvidia blessed binary blobs and is really expensive.
13TOPS, PCIE1x3.0 3.0TOPS/W are promising specs. On an 8GB RPi it might be enough to run a llama3-8B-4QKM model.
I'm so glad I found your channel so awesome and informative videos! keep it up
Got mine ordered.
That toilet detection was hilarious. You can see how it got matched: it looks oval and has a rim and that’s where the logic ends. No consideration of perspective or gravity. lol.
machine learning is not about any considerations, it's just a pre-trained model that understands nothing, but is trying to categorize objects based on the training it was given before.
If you want to shrink this component, you can replace the RJ45 connector with a Mini I/O connector.
I got a coral and the coral hasn't helped at all. Looking forward to working it into my fabric project.
Nope it won't. If you're on a Pi 5 inferencing speed on the CPU is the same performance as the Coral TPU.
Wow, thank you very much for sharing! Always helping! I have a question, I think your website is down!
Thanks - its back up!
+1 liked, for the late night dancing :-), seriously though, $70 for an entry point modest AI chip is pretty impressive. If you consider all the interesting things hobbits have done with PI's over the years, I'm looking forward to see what they come up with, when you add a NPU to the recipe
now all that's remaining is: Frigate support!!
When tried , the hailo causes issues in identification , and aside from the fps , accuracy is very low , is that accuracy lost during conversion ?, i would still use the coral for now
Hi Kevin. Maybe you know a solution for connecting both an Ai Kit hat as well as an ssd on m.2 hat at the same time to a pi 5?
Q could you use a high-def camera and large hard drive and use as a dash camera?
Q: Can I simply attach this module to my raspberry pi, run the same computer vision program I already have, and expect decent improvements? Thank you.
Great question - unfortunately it doesn’t work like that, that programs need to be specifically away of the accelerator and use it to run the ML models
Bit of a misleading title as there was no real comparison between the Google Coral and Hailo 8L other than just a spec sheet side-to-side comparison. I guess it’s just a promo for the RPi AI Kit.
I wanna get into raspberry pi’s and local Ais . Is this called electrical engineering? Is this just a hobby or could I make a living doing this stuff. And without college
AI is part of the programming skillset. You would have to expand upon the hobby question. You can make a living from AI in many’s, but the programming side requires advanced skills and knowledge of algorithms to make a living from. I mean you can make a living in many ways from AI so you’d have to be more specific
I just got a milk-v meles and managed to get it to boot...(it won't power on using the raspberry pi 5 power but will from just a usb from my pc....) I think this RISC-V has potential but the OS support is lacking right now and RISC-V really needs some love from the open source community. I was hoping ubuntu had something already but they only have a broken version for Milk-V mars that the wifi won't even work... Yet... I think hobbiests are going to love this stuff!
What about the cooling, I'm sure both the pi and ai chip will get boiling if you want to build some continuous running high-flops program
Can I use this chip in a standard tensorflow model? If so how? Or is it automatic?
Seems like this proves to be more performant than my Jetson Nano…maybe it’s time to switch back to Pi
Thx for sharing! Apart from object recognition, any other use caes be benefit with edge compute power?
I have struggled SO HARD with the Coral and Raspberry Pi 5. If you are telling me I don't have to create Dockers and virtual environments and whatnot to run older versions of Python I'm IN!!!
Q:Hello..This is cool!... I was wondering if this is enough computer power to work with depth cameras with 3D object detection like the Realsesnse series?...
Same question… not sure if I should go for a Jetson Nano, Orin Nano or a PI5 + AI Kit. I want to run a RealSense + RPLIDAR for object tracking and SLAM.
Looking forward to a tutorial, inferencing custom trained Yolov8 object detection models using this....
Q which case would you put the pi with the hat in? How would you describe the necessity to cool hat or pi?
I'm interested in detecting ArUco markers and getting their pose estimates. Is that something this AI chip can do well? Would you still need to run something like OpenCV on it? I've never used AI specific hardware so not sure how it works.
How well does it perform with LLMs against the base Pi 5?
This chip appears to be low on memory, and not fitting at all for LLMs. You want to look out for Hail-10H though (not released yet, no release date either)
Hi, looks interesting, can this AI Kit also work with a stream from an IP-cam?
man getting one of these is so hard... but it also means that we loose the PCI port to add NVMe / M.2 SSD. Which... why would you run AI without SSD?
We'll see when I get one :P
Can I use this AI kit to speed up my LLMs on the Raspberry Pi 5 with Ollama ?
how are you running llms on a rpi?
Do they cluster stack? Suppose I have an AI project that is 4 different models at once. Can I assign 1 model per pi and make a 4 cluster? I need a LLM that works at least as good as GPT-3, and I need both image(face) recognition, and image generation(from prompt and from fed images, like stable diffusion). Can I use one pi for both image processes, or will I need 5 pi total? I’ve got a cap of 150w because the thing needs to run off a single 200w solar panel(with battery array). Can I get lucky enough to do it all locally without the cloud?
Hi Kevin. Is it posible to make a very small pc not much larger than an Apple TV box. It should be running Linux, and possible to connect, keyboard using USB/Bluetooth with a couple of additional usb ports 😃
Yep - Raspberry Pi 5 plus a tower case (SunFounder have launched one recently)
Thank You Kevin😀
É possivel utilizar essa IA junto com uma camera IP ligado a um NVR? Ou somente com essa mini camera?
Is posible Rpi5 with SSD and Halo for run Asístant and Frigate? What are the correct hat configuration for this pourpose? Thanks
@@galdakaMusic be careful running frigate on a pi as it does not have a compatible encoder and will eat your cpu too much
@@realivanjxI have Home Assitant with Frigate Addon in a Rpi4 with Coral USB. With 4 cameras al 50 Zigbee2mqtt devices without any problem since 4 years. I ask if would be posible migrate to Rpi5, pimoroni nvme dual with SSD and Hailo 8L. Runs Hailo 8L with Frigate??
Is it possible to use this technology to control a UAV so it can be autonomous without GPS? Maybe have it detect objects like buildings mountains lakes etc that it has seen a picture of prior? Where you can use identifiable objects as waypoints that you programmed in prior. I'm a total noob and wonder is there's a name for this idea so I can explore it further?
its called DSLAM, using pictures to map the environment in real time while you navigate it.
so no nvme ssd at the same time as the ai ?
Not with this Hat, but you can use the Pimoroni Duo to have both at the same time.
@@kevinmcaleer28 could we use the Pimoroni Duo and the Hailo accelerator with the same software, etc?
Does openvino work with this ?
(I'd like to use Audacity's openvino tools and Whisper for speech recognition).
Where’s the link for this module?
www.raspberrypi.com/news/raspberry-pi-ai-kit-available-now-at-70/
@@kevinmcaleer28 Do you know any reseller in the EU?
@@eMgotcha77 there’s links in that link
@@eMgotcha77 Pretty sure my friends at Pimoroni ship to the EU!
Thanks, I know. All shops, even Pimoroni, show either nothing or "pre-order, coming soon".
Well i have to exercise patience meanwhile 🙌
I use the PCIe for an NVMe, so how would I add this AI kit, need a splitter?
Is it possible to install both the Acceleator and an SSD?
Yes - you'll need a Pimoroni NVMe Base Duo though
@@kevinmcaleer28 Thank you.
got a few pi's, but just wondering what uses would this have? o.0
That's awesome can you still have an nvme drive
You'll need a Hat that can take 2 NVMe - such as the Pimoroni NVCe Duo
@@kevinmcaleer28 isn't the Raspberry Pi NVME hat stackable? so why cant we have two?
@@bern047 The Raspberry Pi M.2 Hat has one PCIe connector and one M.2 port. The GPIO pins are stackable, but you only get one PCIe connector. Pimoroni Duo has two M.2 ports. There are other Hats available that have multiple PCIe connectors so you can daisy chain them - check out PineBoards website for more info on those.
@@kevinmcaleer28 Thanks, Pimoroni have done it again for me, will order one thanks
Can it run facial recognition model?
Don't know about Ollama, but Hailo, the company behind this M.2 kit (Hailo-8L), the kit has enough processing power to be able to do so.
The coral was noticeable by its absence....?
very cool !!
Unfortunately a lot of AI products look good just looking at the top level specs but then once you have a deeper look they are difficult to use and not all that useful. It will be interesting to see how this ends up working.
I have just bought a cheap jetson orin nano for a project of mine so I won't be buying this anytime soon but it will be interesting to see what people manage to use it for.
This is not like Rabbit or Humane, it actually does the work locally. I’ve made a lot of content about using object detection previously for various projects so this will take that workload off the main CPU freeing it to do other tasks.
Is the 4gb ram raspberry pi 5 good enough for the A.I kit?
Sure is - in fact that’s the one I demoed it on
Can it run on a pi4?
Not without mods. It uses "PCIe Gen 3" connection on Pi5.
@@AerialWaviator Technically it could be made to run on a Compute Module 4-though I haven't tried yet.
@@JeffGeerling Was wondering. Perhaps a custom PCB where both plug-in. ;)
The RPI4 has a PCIe 1x2.0 connected to a USB3 host controller
The CM4 has the PCIe 1x2.0 lane free for the designer to use
Hardware wise, it should work, a compliant pcie 3.0 peripheral should communicate with a 2.0 pcie host at reduced speed. One has to see if the halio pcie silicon interface supports that.
The RPI4 has the BCM2711 SoC, the RPI5 has the BCM2712 SoC. One has to see if the provided Halio driver works on both SoC, or if it is specific for the RPI5 SoC.
I'd really like for the Halio M.2 accelerator to be compatible with CM4, there are lots of CM4 boards out there with a M.2 expansion slot already exposed. e.g. "EdgeBox-RPI-200 equipped a M.2 socket of M KEY type. ONLY 2242 size NVME SSD card is support, NOT mSATA."
Does not work with tflite?
According to the specs, yes!
@@kevinmcaleer28 Hi Kevin, Do you know of any good getting started guides for it? The demos are fine and the hailo site offers very little. I am interested in a guide to loading models and running inference in python so that I can make some particle use of it.
👍👍👍
😎🤖
I guess you have a shitty light there!
Haha
❤❤❤❤❤🎉🎉
But, any actual useful uses of this AI kit?
What actual uses would you find useful?
@kevinmcaleer28 anything besides imagine detection or recognition.
So inspiring to see a successful LGBTQ+ RUclipsr 🏳🌈❤ Kevin!
Ceiling light or mouse…hmmmm
Ceiling light or toilet….hmmmm