Building a Ceph cluster on a bunch of Raspberry Pis was a fun project, and I think perfect for this board! If you're looking for some Pis, they pop up from time to time on rpilocator.com (no affiliation, I just use that site to check up on Pi stock around the world). Thanks especially to KIOXIA for sending the NVMe SSDs to fill in the rest of this board - check out their XG6 and their new XG8 if you need a great, reliable NVMe drive for your computer. I'd been working on this video since a few weeks ago (before the hospitalization), so I was able to get it out for this normal weekly release. I'll probably be taking a break and won't be posting a video next Wednesday-just check back the week after, and in the mean time, go check out Geerling Engineering, my 2nd channel! ruclips.net/user/GeerlingEngineering
I really like your videos because you answer all the relevant questions directly and always get to the point. So thank you, Jeff, for always covering fascinating topics and products. And best greetings to your wife and dad! I really hope you feel better soon!
Can you use something like this as a desktop replacement? I am not clear on if you can configure Linux to use all these Pis as 'one' system transparent to the user and just do normal desktop stuff with it.
AWESOME video! Superb content. But right now, TAKE A BREAK, take care of your health, please. That's the priority. We all are cheering for you, Jeff. God bless you and be better soon. 🙏🏻
Truly impressive video especially given that most of the time your being ill in bed But I have no idea how your keeping Red shirt Jeff under control handcuffs? Video games? A really long leash? So back to my bed to watch you till I fall asleep
Heh... those are literally all the Lite CM4s I have. I have a couple eMMC models too but I typically stick to the Lite modules since they're easier to re-flash (don't have to wait for slow eMMC writes over USB).
@@JeffGeerling I was diagnosed with Collitis the day before you posted your video about your colostomy. Gotta say knowing someone I look up to has the same/similar issue to me healthwise has made it easier to cope.
@@JeffGeerling real clone wars is rhinos like bush=nute gunray invade naboo bush could of prevent europe migrant crisis since obama target Libya Sudan Somalia Syria was originally bush admin rumsfield idea pnac! we can give all medicare by reduce warfronts, no more allowing new daca abusers, force senators to cut their own wealth in gov by half if they want to keep above 70% of current daca population so inflation/living cost stay reasonable wesley clark foreshadow reveal 2000 to 2012 all rig for kill iraq to syria ruclips.net/video/_mrJRHwbVG8/видео.html establishment kill 50 in vegas/portland, thugs attack with stand down cops san jose/charlotte, burn loot several months, sabotage afgan withdraw using russia bounty smear to give taliban equip, crash car in to wisconsin parade thanks to nbc follow jury bus smearing ritten house too ruclips.net/video/UxoL8tHSa7g/видео.html bush 14y ago said add ukraine to nato with nuland f eu coup 2014 support = ruclips.net/video/nTQ3D1a-j20/видео.html current ukraine gov is proxy since cia drew red line just like did in syria earlier arming rebels telling russia not to interfere while zelensky ethnic cleanse donbass region 7y= ruclips.net/video/ta9dWRcDUPA/видео.html ruclips.net/video/IBeRB7rWk_8/видео.html
The quality of your RUclips videos is outstanding. You have such a great mix of technical and communication skills. The things you show (and link to in the descriptions) end up being so approachable. Thank you for putting all this time into the videos for us!
Thank you for marking this as a paid promotion. I've seen too many creators lately that don't count product(that they get to keep, not stuff they have to send back) as payment. Cash didn't change hands, but they were still paid. Also, I love that trick with the sticky note. I wouldn't have thought of that, but it's brilliant for dealing with so many small microsd cards.
Haha yeah I started doing that about the fifth time I lost track of microSD cards on a cluster build... it's perfect since I always have a pile of Post-Its nearby!
An interesting alternative, but if I were to try this I think I'd go with the Turing 2 for the reasons you mentioned. I am glad to see that you are feeling better. I know from experience that gastric diseases are no fun. Thank you for all you do!
Good news: no individual power adapters and cables needed. Bad news: the PSU that powers the board is now a single point of failure. Thanks for the Ceph tutorial, Jeff. I think I now have the intestinal fortitude to try a Ceph deployment myself. And BONUS - what a great hands-on opportunity to really learn and understand Ansible.
You don't need 5 power supplies when clustering normal Pi4s, just one feisty 5V supply, which you can also then feed to the Ethernet switch. Meanwell comes highly recommended, and they have a 90W/18A model. You might want to 3D print a MicroSD card puller that you can clip onto the cards to retrieve them. I've seen a bunch of designs of those floating around.
@@JeffGeerling It is a fun game but losing it sucks and even when winning you then get to play the second round of the game which is named 'Desperately search the floor on your hands and knees because you know that if you dont find the pin you will feel it in three days when you step on it barefoot...' So many times.....
It's way quicker to cut than to desolder. Desolder on 100 boards and I'm pretty sure you will be quite tired. Even lots of massproduced motherboards has cut pins.
@@perwestermark8920 I've never seen a mass-produced motherboard with snipped pins, do you have an example? Desoldering is usually quicker though, touch the pin with the soldering iron and pull it out just like that, no need to line up cutters and no risk of leaving a pin loose on the motherboard ready to short circuit everything when an unsuspecting customer connects power...
Great to have you well enough to post Jeff. I do feel for companies that have developed such cool stuff for the rpi and especially the cm4. I don’t think I’ve actually seen a cm4 available for sale through an Australian based website at all. Someday!
Man it is good to see you up and around and cranking out these great videos again. I hope you are feeling better. You must be feeling some better since you’re up and around at least. That’s a great thing. Keep cranking man
One of the features of a cluster is a reduction in the single-point-of-failure elements. For home use, you can't do anything about AC power, but your original cluster at least had separate power adapters and ethernet ports, although just a single switch. This board appears to have more single-point-of-failure components.
Happy to see you're back and hope you recovered full. Were waiting on your video and it's quite interesting to see how your RPI cluster have evolved. I will dive more in ceph/ansible using VMs instead...
I just randomly stumbled upon this video and just realized you are "The" geerlingguy. Holy frick mate, your tools have made my life so much easier. Thanks for all the hard work. Great video.
It is definitely the number one issue with these projects; you either have to spend a bunch of money or have a huge amount of patience to get Pis anymore :(
I like your clustering vids (touring pie, deskpie, conventional with router poe, etc). The parts described in the des just expensive when shipping to Indonesia (vat, shipping cost). Thats really put me into tears
1. I am SUPER happy to see you producing content again. It implies you are feeling better! 2. This board appears to be really useful for learning ceph, or other technologies. 3. This board is almost useless because it is the foundation for rpi boards that are mostly unavailable. 4. I have been having some really good luck with the "Le Potato" to fill most of the needs for raspberry pi's. I don't always need the power of an rpi4 and the "potato" is just a bit more power than a 3b, and at a slightly lower power consumption so almost any phone charger will now work very well. 5. Maybe with the whole world going into a recession, maybe these boards will be available soon?
Great to see you back to making videos. Pace yourself . Thanks to your family for helping you and weathering this episode. You are my favorite tech youtuber. The amount you give back to the community is awesome. Keep up the great work
It's not fair! I've been trying to buy just one CM4 for nearly a year now and it's out of stock everywhere. Now here come's Jeff with his SIX CM4s on a single board! 😭
Ah but I ordered four of these CM4s in October 2020, and the other two in June 2021, heh... they took months to come in! I have been seeing 2 and 4 GB CM4 modules coming in stock every week or so, if you need one make sure to get notifications from rpilocator!
@@JeffGeerling Yeah, I'm keeping an eye on that site. Hopefully there'll be some UK stock soon. It's super frustrating seeing all these amazing carrier boards on your channel and not being able to make use of any of them. 🙂
I can see this working really well in distributed TV editing storage, particularly for outside broadcast or mobile facilities. Low power requirements, easily configured to access 6TB at time. The whole thing, including monitor, keyboard and mouse could be put in a flight case and carried to the next job. The only downside is the gigabit connecting. 150 minutes minimum per terabyte is slow but that's where the price and portability come in. Two or three cases per reporting team would keep everything flowing nicely.
Am glad to see an upload. I know this is from before but I hope you are taking care of yourself and that you are getting better. Get well soon dude! We miss you! :) 🤗🤗🤗👍👍👍❤️❤️❤️
I'm pretty sure I can get my normal heatsinks to fit, I would just need to use a different set of spacers and screws so I can still tighten them directly to the board (the inserts on the board are threaded, I believe M2.5).
You weren't kidding when you said it had it's trade offs. Honestly I'd prefer the Turing Pi, as compatibility with Jetson platform, and ATX power compatibility would make it a great fit in home-labs; my biggest gripe with the Turing is how IO is handled by specific slots rather than one master.
Finally getting to what I wanted with Ceph. Really cool I knew you would come around. HA!!! I have some reading to do with Ceph; I did not know it existed.
Hopefully I'll get some more time to dive into Ceph; it impressed me with how simple it was to get going, and the concept is similar to Gluster (which I'm more familiar with), but I need a bit more time with it before I'd be comfortable going deeper in a video!
Thanks Jeff. Great video. I came across your post about two months ago and really enjoy the content and your presentation style! Loved your Turning Pi videos and now this one too. I am waiting on my Turing Pi 2 to ship and I'm going to order this board too! Thanks again!
This is yet another step forward, I love it. Maybe in the future we'll see more home super computers. I'm glad you recorded this earlier, hopefully you're still taking it easy. And I'll never stop using the SI prefixes to refer to 1024 increments of bytes. I don't care if hard drive manufacturers cheat, computers are special and it makes no sense to make up new terms this late in the game, aside from the fact that they sound dumb.
@@tomservo5007 Not really, considering how they were built in the past. It's just that processors have gotten so much better now that we, most of us anyway, have as much computing power in our pockets as supercomputers did in the 70's.
@@anon_y_mousse And as a note, this board's linpack scores would put it somewhere in the Top500 list from around the year 2000... so it just depends on your definition of 'Supercomputer' ;)
Much better! Though still taking it easy right now, as part of that is due to some of the medications I'm on... always hard to figure out how much improvement comes from *actual* improvement in underlying health!
Even without clustering I can see a lot of fun server/headless use scenarios for having 6 computers attached to the network while taking very little power and physical space. I can even see uses for having the two ethernet ports such as having them connected to two separate switches so two different networks can each talk to those servers but nor each other directly. Think about things like having one pi dedicated to just doing git pulls and nightly builds of software, one as a media server for the home, the one with HDMI out being for emulation or a media player or a low end desktop, one dedicated to testing software builds before deploying to other machines separate from the one that just auto build nightlies for you, and two for a low end server with a redundant backup in case of a software config screwup or data corruption.
Those port act as a dumb switch, so connecting them to 2 switch will end like you used a wire between the switch NOT 2 separates networks Basictly, the board is a 8 port dumb switch, where 6 are connected to the pi.
This is super neat! As a testbed, this makes so much sense. Or just running a small homeserver - its really neat! However, I am still waiting for just one CM4 on a miniITX board - I have an old thin MiniITX board lying around here and I would love to put a CM4 into that to use it to run HomeAssistant and friends. The board you showed off might actually fit, which is kinda insane. xD But the case's built-in PSU just won't do 100W...
Fascinated by the idea of something like this or the Turing Pi for a little home cluster to do some Docker experiments. This beats the Turing Pi by virtue of being available but on the other hand I'm not sure if it is worth going all-in on six CM4s, six SD cards and six m.2 drives vs just going to state surplus and picking up an old pc or two.
Thanks, Jeff. I can see this being used for a micro computational cluster for use at the edge. I like to take low cost clusters and then benchmark them to see if/where they would have shown up on the Top 500 Supercomputers list. That is, what year. For example, your configuration might result in a “score” of 1998 - meaning that 6-node cluster would have been a top 500 supercomputer back in 1998.
The main reason to have 2 ports at the back is the chip supports 8 ports, and 6 are used by Pis, so 2 at the back are natural way to have it. It allows daisy chaining multiple clusters for example. Useful if you do not have much network traffic.
2 года назад
I don't know anything about advanced blending or programming, but I find this type of video interesting. Go figure...
That unmanaged network switch is a real killer for the utility of this unit outside of personal users/home labs. It would have been trivial to add a smartswitch with basic management, or a managed chip from realtek with a serial connection to the first pi or a console port even.
Protip on the SD cards. Just put little scotch tape tails on em. Makes em easy to pull out of tight spots. Just don't do that in an application where they may get snagged and pulled accidentally.
You could use multipath TCP for getting more speed, and as it's CEPH and storage you could use SMB3 multichannel feature which works with at least Windows 10 and modern Samba. Neither of these solutions require a managed switch.
But both of these require multipath connections to the nodes. Here, each Pi has only 1 Ethernet connection and the switch has two ports exposed. Connect those two ports to another switch and you have a loop, causing a smarter switch to disable one of the ports or when using a not too smart switch will just lock up the network... So SMB multipath won't work here unfortunately, that is why it is hard to understand why they chose an unmanaged switch...
@@someguy4915 Duh that was a Homer Simpson moment. However with a suitable managed switch you could put down MAC based ACL's that would mask half the PI's down one port and half down the second port, to get more upstream speed. That should shut down the loop, but yeah it's slightly wacho.
@@jonathanbuzzard1376 True but even then you'd need a bunch of routing etc. to make sure that traffic sent on Port 1 also returns through Port 1 etc. which while certainly possible is just a bit of a headache when a simple managed switch would fix all of that and provide so much more functionality. Besides LACP port bonding you'd also get VLAN support, now you could just have the Pi's spit through VLAN tags and hope that the buffer of the switch is big enough, usually this does work, but it is unsupported and a great way to get very complex network issues (don't ask how I learned that lesson... :P)
I seem to recall that a netgear (dumb) switch that I owned (may still be in a box here some where) actually supported link agregation, even if it didn't have management or vlan support. So it's possible that getting link aggregation on the board may just be a firmware update, but I'm kind of fond of the vlan support as it allows me to move management of devices off of the vlan that the user data is running over. Hopefully the manufacturer will get that all worked out.
Switches these days will usually detect multiple connections and disable one if there's not link aggregation. Otherwise you get network storms with redundant broadcast packets getting received and sent back out again through a loop. At least that's how I remember it, it's been far too long since I did any certification courses.
I do like SFP+, though just having an SFP+ cage wouldn't give faster speeds, the switch chip would also need an upgrade. 2.5G could be realistic but going to 5 or 10G on this board design might have issues with heat or signal routing. The other difficulty with this board in particular is there's not enough physical room for SFP+; you'd have to take out one of the Pis to do it.
@@JeffGeerling Oh, yes I see now, you are right. The advantage with a SFP+ slot is that it can run on less power than a 10GE copper connection. But perhaps one or two 2,5GE slots would be good? The Intel I225 should perhaps fit well? It had issues in the first two revisions. But yes the whole switch thing is an issue as well.
@@JohnAngelmo But that Intel is a NIC, not a switch, you'd need to replace the entire Realtek switch chip for something more useful (managed and 2.5-10GbE) before adding SFP+ has any benefit.
My guess is the signal routing would've been a bit more complex trying to get all the cards lined up along an edge. It could've increased the complexity of the board design and possibly required more PCB layers, which would increase the overall cost (and reduce margins). One nice thing I didn't mention is you can actually boot off the NVMe drives, so you technically don't need to use the microSD card slots at all, even if you have Lite modules.
Building a Ceph cluster on a bunch of Raspberry Pis was a fun project, and I think perfect for this board! If you're looking for some Pis, they pop up from time to time on rpilocator.com (no affiliation, I just use that site to check up on Pi stock around the world).
Thanks especially to KIOXIA for sending the NVMe SSDs to fill in the rest of this board - check out their XG6 and their new XG8 if you need a great, reliable NVMe drive for your computer.
I'd been working on this video since a few weeks ago (before the hospitalization), so I was able to get it out for this normal weekly release. I'll probably be taking a break and won't be posting a video next Wednesday-just check back the week after, and in the mean time, go check out Geerling Engineering, my 2nd channel! ruclips.net/user/GeerlingEngineering
I really like your videos because you answer all the relevant questions directly and always get to the point. So thank you, Jeff, for always covering fascinating topics and products. And best greetings to your wife and dad! I really hope you feel better soon!
Can you use something like this as a desktop replacement? I am not clear on if you can configure Linux to use all these Pis as 'one' system transparent to the user and just do normal desktop stuff with it.
AWESOME video! Superb content. But right now, TAKE A BREAK, take care of your health, please. That's the priority. We all are cheering for you, Jeff. God bless you and be better soon. 🙏🏻
I'd love to see some of the points per day it could generate on folding@home if that is possible.
Truly impressive video especially given that most of the time your being ill in bed
But I have no idea how your keeping Red shirt Jeff under control handcuffs? Video games? A really long leash?
So back to my bed to watch you till I fall asleep
If anyone is wondering why there are no rpis in stock its because Jeff has them ALL!
Heh... those are literally all the Lite CM4s I have. I have a couple eMMC models too but I typically stick to the Lite modules since they're easier to re-flash (don't have to wait for slow eMMC writes over USB).
No doubt!!!!
@@JeffGeerling totally sus how he has so many LoL he works for RPi maybe 🤔
@@JeffGeerling That's what someone who bought *all* the Pis would say! :D
dont forget red "shirt" jeff
It is good to see you back healthy. Hope you are fully recovered Jeff!
Like always awesome content +1.
Not quite fully recovered, but we'll get there!
@@JeffGeerling 👍🏼Hang in there! Hope it's (relatively) easy for you and family! 😎✌🏼
@@JeffGeerling I was diagnosed with Collitis the day before you posted your video about your colostomy. Gotta say knowing someone I look up to has the same/similar issue to me healthwise has made it easier to cope.
@@JeffGeerling real clone wars is rhinos like bush=nute gunray invade naboo
bush could of prevent europe migrant crisis since obama target Libya Sudan Somalia Syria was originally bush admin rumsfield idea pnac!
we can give all medicare by reduce warfronts, no more allowing new daca abusers, force senators to cut their own wealth in gov by half if they want to keep above 70% of current daca population so inflation/living cost stay reasonable
wesley clark foreshadow reveal 2000 to 2012 all rig for kill iraq to syria
ruclips.net/video/_mrJRHwbVG8/видео.html
establishment kill 50 in vegas/portland, thugs attack with stand down cops san jose/charlotte, burn loot several months, sabotage afgan withdraw using russia bounty smear to give taliban equip, crash car in to wisconsin parade thanks to nbc follow jury bus smearing ritten house too
ruclips.net/video/UxoL8tHSa7g/видео.html
bush 14y ago said add ukraine to nato with nuland f eu coup 2014 support =
ruclips.net/video/nTQ3D1a-j20/видео.html
current ukraine gov is proxy since cia drew red line just like did in syria earlier arming rebels telling russia not to interfere while zelensky ethnic cleanse donbass region 7y=
ruclips.net/video/ta9dWRcDUPA/видео.html
ruclips.net/video/IBeRB7rWk_8/видео.html
Great to see you are doing good Jeff
The quality of your RUclips videos is outstanding. You have such a great mix of technical and communication skills. The things you show (and link to in the descriptions) end up being so approachable.
Thank you for putting all this time into the videos for us!
Thank you for marking this as a paid promotion. I've seen too many creators lately that don't count product(that they get to keep, not stuff they have to send back) as payment. Cash didn't change hands, but they were still paid.
Also, I love that trick with the sticky note. I wouldn't have thought of that, but it's brilliant for dealing with so many small microsd cards.
Haha yeah I started doing that about the fifth time I lost track of microSD cards on a cluster build... it's perfect since I always have a pile of Post-Its nearby!
An interesting alternative, but if I were to try this I think I'd go with the Turing 2 for the reasons you mentioned.
I am glad to see that you are feeling better. I know from experience that gastric diseases are no fun. Thank you for all you do!
Good news: no individual power adapters and cables needed. Bad news: the PSU that powers the board is now a single point of failure.
Thanks for the Ceph tutorial, Jeff. I think I now have the intestinal fortitude to try a Ceph deployment myself. And BONUS - what a great hands-on opportunity to really learn and understand Ansible.
The switch and ethernet cable are also single points of failure.
This board is cool and I'm sure it's good for something, I just don't know what...
Hey! I just found out that I need to learn about Ansible for work. I'm absolutely going to check out your videos, on the company dime. Thanks so much.
They're a great set of videos, and is how I learned Ansible. 👍🐧
You don't need 5 power supplies when clustering normal Pi4s, just one feisty 5V supply, which you can also then feed to the Ethernet switch. Meanwell comes highly recommended, and they have a 90W/18A model.
You might want to 3D print a MicroSD card puller that you can clip onto the cards to retrieve them. I've seen a bunch of designs of those floating around.
Hey Jeff.
Glad to see you uploaded again. Hopefully you are feeling better. I'm sorry things have been rough for you. Take care.
Thank you!
5:19 I don't know why they're snipping off the pin. it's a really easy task to desolder the single pin and make it look good.
Probably they enjoy the challenge of the "don't let the pin fly into your eyeball when you snip it" game!
Red shirt jeff has to have his moments too
@@JeffGeerling It is a fun game but losing it sucks and even when winning you then get to play the second round of the game which is named 'Desperately search the floor on your hands and knees because you know that if you dont find the pin you will feel it in three days when you step on it barefoot...'
So many times.....
It's way quicker to cut than to desolder. Desolder on 100 boards and I'm pretty sure you will be quite tired. Even lots of massproduced motherboards has cut pins.
@@perwestermark8920 I've never seen a mass-produced motherboard with snipped pins, do you have an example?
Desoldering is usually quicker though, touch the pin with the soldering iron and pull it out just like that, no need to line up cutters and no risk of leaving a pin loose on the motherboard ready to short circuit everything when an unsuspecting customer connects power...
Glad you managed to get to this, hope you're feeling better!
Much better! Going to keep on with my break though, so next video will likely come in two weeks instead of one!
Take your time, health is always more important and the videos are worth the wait
Happy to see your up and aboard again. I'm waiting for my Turing Pi2 but no CM4s.
Great to have you well enough to post Jeff. I do feel for companies that have developed such cool stuff for the rpi and especially the cm4. I don’t think I’ve actually seen a cm4 available for sale through an Australian based website at all. Someday!
spoiler it can't well I would hope not it's not any where near the 12k price tag
Man it is good to see you up and around and cranking out these great videos again. I hope you are feeling better. You must be feeling some better since you’re up and around at least. That’s a great thing. Keep cranking man
One of the features of a cluster is a reduction in the single-point-of-failure elements. For home use, you can't do anything about AC power, but your original cluster at least had separate power adapters and ethernet ports, although just a single switch. This board appears to have more single-point-of-failure components.
4:40 Love that SD card management thingy.
Heh, post it notes have built in sticky pads to hold the cards in place!
Glad to see you are feeling well enough to post again!
Happy to see you're back and hope you recovered full. Were waiting on your video and it's quite interesting to see how your RPI cluster have evolved. I will dive more in ceph/ansible using VMs instead...
I just randomly stumbled upon this video and just realized you are "The" geerlingguy. Holy frick mate, your tools have made my life so much easier. Thanks for all the hard work. Great video.
Haha, thanks, glad you found the channel :)
Hey Jeff. Glad to see you finally got this one out. 👍The shortage of Pi's however is sadly becoming a real problem for innovative projects like this.
It is definitely the number one issue with these projects; you either have to spend a bunch of money or have a huge amount of patience to get Pis anymore :(
Tell me about it, I'm waiting since January for my RBPi4 8GB and I actually wanted to buy 7 to replace my x86 server...
I hope you are feeling better man you are my go to for anything Pi related !
Thank you for showing us the Ansible parts! I found that very helpful.
"DNS is the root of all problems" I would ask for his autograph because of his shirt alone :)
but why., dns is good
I like your clustering vids (touring pie, deskpie, conventional with router poe, etc). The parts described in the des just expensive when shipping to Indonesia (vat, shipping cost). Thats really put me into tears
1. I am SUPER happy to see you producing content again. It implies you are feeling better!
2. This board appears to be really useful for learning ceph, or other technologies.
3. This board is almost useless because it is the foundation for rpi boards that are mostly unavailable.
4. I have been having some really good luck with the "Le Potato" to fill most of the needs for raspberry pi's. I don't always need the power of an rpi4 and the "potato" is just a bit more power than a 3b, and at a slightly lower power consumption so almost any phone charger will now work very well.
5. Maybe with the whole world going into a recession, maybe these boards will be available soon?
Nice to see you over your sickness, you always make great videos!
Good to see you back and hope you are doing well. Also; leave some RPIs for us regular folks. Great videos as always!
Welcome back Jeff, wonderful video!
Great to see you back to making videos. Pace yourself . Thanks to your family for helping you and weathering this episode. You are my favorite tech youtuber. The amount you give back to the community is awesome. Keep up the great work
Glad to see you "full of energy", and the cutting edge news.
It's not fair! I've been trying to buy just one CM4 for nearly a year now and it's out of stock everywhere. Now here come's Jeff with his SIX CM4s on a single board! 😭
Ah but I ordered four of these CM4s in October 2020, and the other two in June 2021, heh... they took months to come in!
I have been seeing 2 and 4 GB CM4 modules coming in stock every week or so, if you need one make sure to get notifications from rpilocator!
@@JeffGeerling Yeah, I'm keeping an eye on that site. Hopefully there'll be some UK stock soon. It's super frustrating seeing all these amazing carrier boards on your channel and not being able to make use of any of them. 🙂
BerryBase DE has a load of CM4s in stock, maybe give that a look, although I suppose shipping would be a pain
@@alexbissessur5013 Thanks. Do you know if they would deliver to the UK?
@@hadron2 Sadly you have to be a member of their club and I've not found out how you join it, so it won't let you add to basket ...
So glad to hear you're doing better! Amazing video as always man
Every video is a commercial from this guy. He comes off really ingenuous.
I can see this working really well in distributed TV editing storage, particularly for outside broadcast or mobile facilities. Low power requirements, easily configured to access 6TB at time. The whole thing, including monitor, keyboard and mouse could be put in a flight case and carried to the next job. The only downside is the gigabit connecting. 150 minutes minimum per terabyte is slow but that's where the price and portability come in. Two or three cases per reporting team would keep everything flowing nicely.
Am glad to see an upload. I know this is from before but I hope you are taking care of yourself and that you are getting better.
Get well soon dude! We miss you! :) 🤗🤗🤗👍👍👍❤️❤️❤️
Finally we in Germany get to have cm4s! They once hat the "rich kid" cm4 like a month ago with everything maxxed!
Great build and review, Jeff. I hope you're taking it easy though. We can wait longer between videos if it means you getting healthier!
just glad your feeling better
Thank you so much Jeff! In just this post you let me discover 3 solutions I ignored. Great post.
Now that is a supercomputer, would be even better if the SOCs and other on-Pi chips had heatsinks.
I'm pretty sure I can get my normal heatsinks to fit, I would just need to use a different set of spacers and screws so I can still tighten them directly to the board (the inserts on the board are threaded, I believe M2.5).
If they wanted fast and cheaper they wouldn't be using RPis to begin with.
@@trinidad17 Nvidia jetsons? Pine64 whatevers?
Or perhaps use a submerged liquid cooler like the one just shown on LTT channel
it's still not a supercomputer. even with a lot more boards, the overall limitations are too many for it to reach such levels of performance.
I see your feeling better Jeff. I hope you stay that way for bit.
You weren't kidding when you said it had it's trade offs. Honestly I'd prefer the Turing Pi, as compatibility with Jetson platform, and ATX power compatibility would make it a great fit in home-labs; my biggest gripe with the Turing is how IO is handled by specific slots rather than one master.
Finally getting to what I wanted with Ceph. Really cool I knew you would come around. HA!!! I have some reading to do with Ceph; I did not know it existed.
Hopefully I'll get some more time to dive into Ceph; it impressed me with how simple it was to get going, and the concept is similar to Gluster (which I'm more familiar with), but I need a bit more time with it before I'd be comfortable going deeper in a video!
@@JeffGeerling I'd have to see if it supports a fail over head node.
Thanks Jeff. Great video. I came across your post about two months ago and really enjoy the content and your presentation style! Loved your Turning Pi videos and now this one too. I am waiting on my Turing Pi 2 to ship and I'm going to order this board too! Thanks again!
Still waiting on my Turing Pi 2 board, but this board arrives in two weeks. Can't wait! Just watched your video again to help keep me calm. LOL
I hadn't heard of Ceph before, super cool combo of tech!
Hope you fully recover soon!
Looks like Jeff is feeling better! Thank goodness.
This is yet another step forward, I love it. Maybe in the future we'll see more home super computers. I'm glad you recorded this earlier, hopefully you're still taking it easy. And I'll never stop using the SI prefixes to refer to 1024 increments of bytes. I don't care if hard drive manufacturers cheat, computers are special and it makes no sense to make up new terms this late in the game, aside from the fact that they sound dumb.
Whenever I say mebibyte, I think of a little toddler trying to say 'megabyte' :D
'supercomputer' , that's stretching the term a wee bit
@@tomservo5007 Not really, considering how they were built in the past. It's just that processors have gotten so much better now that we, most of us anyway, have as much computing power in our pockets as supercomputers did in the 70's.
@@JeffGeerling I wasn't going to go there, but you said it, not me.
@@anon_y_mousse And as a note, this board's linpack scores would put it somewhere in the Top500 list from around the year 2000... so it just depends on your definition of 'Supercomputer' ;)
so encouraging watching all these people with their bunch of pis... I can't even get one
This video forces you to want to purchase more Pi's. I love this board soooo much!
Another good 'un, Geerling!
Certainly hope you are doing much , much better!
Look forward to the next installment of JG!
Good luck getting enough CM4's !! Even if you find some available..there are often order limits of 1 per customer.
Wow! Awesome video! Thank you very much for this.
The last pi home server you will ever need. Gigabit lan, 24 Gb of ram, and 6 ssd’s!!!
Just wish it exposed more GPIO-it'd be cool to be able to integrate a HAT into the platform too!
@@JeffGeerling we just may need a bigger board and more pcie lanes for that.
But it's not. It's 6 servers on a single board. Resources cannot be be summed up like that.
I like the part where he says "It's clusterin' time!", and then he clustered all over those Pis
morbius gone F4
This shit is violently unfunny
Glad to see you up and running.😁
I'm happy that your back.
Nice to see you up and about.
Glad to see you back. 💯
Awesome, I loved the W.O.P.R. reference, I just watched that movie for the umpteenth time not too long ago.
Shall we play a game?
Now i understand what monitoring system i do for. Interesting work have my older colleagues.
Oh, Christmas can’t come soon enough🥰
Thanks Jeff! I am going to get on of these. Thanks for leading the way and this is so cool!. Oh, and I hope you are feeling better!
Much better! Though still taking it easy right now, as part of that is due to some of the medications I'm on... always hard to figure out how much improvement comes from *actual* improvement in underlying health!
I appreciate all your content, best of luck.
great content as usual. It's also good to see that you're feeling better.
It's a different board, you are looking good. I hope everything has become more manageable and your health is back.
Keep baking new Pies.
That Ansible wow made my day.
Even without clustering I can see a lot of fun server/headless use scenarios for having 6 computers attached to the network while taking very little power and physical space. I can even see uses for having the two ethernet ports such as having them connected to two separate switches so two different networks can each talk to those servers but nor each other directly.
Think about things like having one pi dedicated to just doing git pulls and nightly builds of software, one as a media server for the home, the one with HDMI out being for emulation or a media player or a low end desktop, one dedicated to testing software builds before deploying to other machines separate from the one that just auto build nightlies for you, and two for a low end server with a redundant backup in case of a software config screwup or data corruption.
Those port act as a dumb switch, so connecting them to 2 switch will end like you used a wire between the switch NOT 2 separates networks
Basictly, the board is a 8 port dumb switch, where 6 are connected to the pi.
Glad to see youre doing better!
This is super neat! As a testbed, this makes so much sense. Or just running a small homeserver - its really neat!
However, I am still waiting for just one CM4 on a miniITX board - I have an old thin MiniITX board lying around here and I would love to put a CM4 into that to use it to run HomeAssistant and friends. The board you showed off might actually fit, which is kinda insane. xD But the case's built-in PSU just won't do 100W...
This board only really needs 25W... maybe 30-40W total for overhead. That is, unless you use some really high-powered NVMe drives!
Fascinated by the idea of something like this or the Turing Pi for a little home cluster to do some Docker experiments. This beats the Turing Pi by virtue of being available but on the other hand I'm not sure if it is worth going all-in on six CM4s, six SD cards and six m.2 drives vs just going to state surplus and picking up an old pc or two.
Yet another great video. Hope you're feeling better, mate.
2:50 The board has it's own little PMU -> The board has ITS own little PMU.... for the rest great videoarticle!
Love your channel. Keep up the great work.
Thanks, Jeff. I can see this being used for a micro computational cluster for use at the edge. I like to take low cost clusters and then benchmark them to see if/where they would have shown up on the Top 500 Supercomputers list. That is, what year. For example, your configuration might result in a “score” of 1998 - meaning that 6-node cluster would have been a top 500 supercomputer back in 1998.
Great video as always, thanks Jeff !
For a fan controller you could fit a PWM 4-Wire Fan Temperature Controller board internally.
The main reason to have 2 ports at the back is the chip supports 8 ports, and 6 are used by Pis, so 2 at the back are natural way to have it. It allows daisy chaining multiple clusters for example. Useful if you do not have much network traffic.
I don't know anything about advanced blending or programming, but I find this type of video interesting. Go figure...
6:58 "How am I going to manage six Raspberry Pi's?"
We all knew exactly where you were going with this, Mr. Ansible master.
Why do raspberry pi videos bang more than most other computer videos... It's insane
That unmanaged network switch is a real killer for the utility of this unit outside of personal users/home labs. It would have been trivial to add a smartswitch with basic management, or a managed chip from realtek with a serial connection to the first pi or a console port even.
Jeff: Im feeling a bit sick and I need to take some time off.
Also Jeff: *dropping one video after another*
👏🏼♥️💩💨
Heh this video was close to ready a couple weeks ago when I wound up in the hospital. It'll be a bit before I'm back on a normal schedule :(
I like this guy, he doesn't hide that 50% of his appeal for this thing are the beepy boopy lights
HYPE HYPE HYPE!
Good to see you back at it Jeff
Jeff, single-handedly keeping the Raspberry-Pi industry afloat
Can't wait when 8Gb back in stock to build a similar cluster.
God bless you jeff, praying for you buddy,we appreciate everything you do , your health comes first, your pretty awesome to share with others🤓
Protip on the SD cards. Just put little scotch tape tails on em. Makes em easy to pull out of tight spots. Just don't do that in an application where they may get snagged and pulled accidentally.
You could use multipath TCP for getting more speed, and as it's CEPH and storage you could use SMB3 multichannel feature which works with at least Windows 10 and modern Samba. Neither of these solutions require a managed switch.
But both of these require multipath connections to the nodes.
Here, each Pi has only 1 Ethernet connection and the switch has two ports exposed.
Connect those two ports to another switch and you have a loop, causing a smarter switch to disable one of the ports or when using a not too smart switch will just lock up the network...
So SMB multipath won't work here unfortunately, that is why it is hard to understand why they chose an unmanaged switch...
@@someguy4915 Duh that was a Homer Simpson moment. However with a suitable managed switch you could put down MAC based ACL's that would mask half the PI's down one port and half down the second port, to get more upstream speed. That should shut down the loop, but yeah it's slightly wacho.
@@jonathanbuzzard1376 True but even then you'd need a bunch of routing etc. to make sure that traffic sent on Port 1 also returns through Port 1 etc. which while certainly possible is just a bit of a headache when a simple managed switch would fix all of that and provide so much more functionality.
Besides LACP port bonding you'd also get VLAN support, now you could just have the Pi's spit through VLAN tags and hope that the buffer of the switch is big enough, usually this does work, but it is unsupported and a great way to get very complex network issues (don't ask how I learned that lesson... :P)
This is great for folk like me who don't really do hardware.
It'd be neat if you could double up two boards as "layers" and have them both interact with one another, so you can fit 12 pi's in the case
I seem to recall that a netgear (dumb) switch that I owned (may still be in a box here some where) actually supported link agregation, even if it didn't have management or vlan support. So it's possible that getting link aggregation on the board may just be a firmware update, but I'm kind of fond of the vlan support as it allows me to move management of devices off of the vlan that the user data is running over. Hopefully the manufacturer will get that all worked out.
Switches these days will usually detect multiple connections and disable one if there's not link aggregation. Otherwise you get network storms with redundant broadcast packets getting received and sent back out again through a loop. At least that's how I remember it, it's been far too long since I did any certification courses.
Yeah; that seemed to be the case here, so best case is you just get a redundant network path.
Love that WOPR ref.
Hmm, I guess I would prefer a SFP+ slot for networking so you can use 1GE, 2,5GE, 5GE or 10GE depending on need if you can get 1GE from each pie.
I do like SFP+, though just having an SFP+ cage wouldn't give faster speeds, the switch chip would also need an upgrade. 2.5G could be realistic but going to 5 or 10G on this board design might have issues with heat or signal routing.
The other difficulty with this board in particular is there's not enough physical room for SFP+; you'd have to take out one of the Pis to do it.
@@JeffGeerling Oh, yes I see now, you are right. The advantage with a SFP+ slot is that it can run on less power than a 10GE copper connection. But perhaps one or two 2,5GE slots would be good? The Intel I225 should perhaps fit well? It had issues in the first two revisions. But yes the whole switch thing is an issue as well.
@@JohnAngelmo But that Intel is a NIC, not a switch, you'd need to replace the entire Realtek switch chip for something more useful (managed and 2.5-10GbE) before adding SFP+ has any benefit.
It's like these guys have never had an SD card barf its guts. What a crazy place to put the port.
My guess is the signal routing would've been a bit more complex trying to get all the cards lined up along an edge. It could've increased the complexity of the board design and possibly required more PCB layers, which would increase the overall cost (and reduce margins).
One nice thing I didn't mention is you can actually boot off the NVMe drives, so you technically don't need to use the microSD card slots at all, even if you have Lite modules.
@@JeffGeerling oh well, that's ideal then. I trust my nvme at least 3500% more than an SD card.
It's nice to see "Excited Jeff" again. The last video made want to send him a subscription to Harry and David's Chicken Soup of the Month.