STH Project TinyMiniMicro Home Lab Revolution Introduction
HTML-код
- Опубликовано: 12 июл 2020
- See the main site article here: www.servethehome.com/introduc...
STH Project TinyMiniMicro is set to revolutionize the home lab segment with clusters of high-quality, quiet, low power, and inexpensive nodes. We have purchased tons of second-hand Lenovo Tiny, HP Mini, and Dell Micro nodes to see what each offers in terms of power, performance, noise, and expansion. In this series, we are going to cover the nodes themselves, but also important aspects of the lifecycle including purchasing second-hand units.
By purchasing so many, we have found a number of variations and deviations from what we would have otherwise expected. In the process, we found that these units offer enormous value, but there were a lot of pitfalls (and nice surprises) to purchasing so many used units. Our goal over the series is to help you understand what to look for so you can learn from our mistakes purchasing a large population of these.
Project TinyMiniMicro on the STH forums for more detail as well: forums.servethehome.com/index...
Other STH References in this piece
- Of BBQ and Virtualization (Video): • Of Virtualization and BBQ
- Of BBQ and Virtualization (Article): www.servethehome.com/of-bbq-a...
- HPE ProLiant MicroServer Gen10 Plus Review: www.servethehome.com/hpe-prol...
- HPE ProLiant MicroServer Gen10 Plus Ultimate Customization Guide (Video): • Ultimate Guide for HPE...
- HPE ProLiant MicroServer Gen10 Plus Ultimate Customization Guide (Article): www.servethehome.com/hpe-prol... 
Наука
I believe I got onto this channel when you did a review of the older HP/AMD desktops, as a cheap fw/router. I kept at it and ended up buying a HP Deskpro 600 SFF desktop for my new OPNsense firewall. I paid a total of $300 for a desktop with a i5-7600, 1TB SATA drive, 8 gig RAM, and an add in 4 port Ethernet card.
Great value!!
I can't wait to see what this series has in store. I inherited 4 HP t620s and 4 HP t630 thin clients that I plan on running in a Proxmox cluster and can't wait to see what you do with these little beasts!
I love these things. I've used them running Kodi in the past, and they're great for it. Now, I have just the one and it's running Sangoma FreeBBX; never missed a beat.
Wish that more of these had dual Lan... Would make a great base for an Untangle or Pfsense firewall
I'm using USB ethernet adapter for this purpose under esxi with native driver.
I measured bandwidth between two usb nic by using iperf and got average of ~950mbit/sec unidirectional.
@@davidbeszeda6894 very nice ! Most USB nic's seem to be Realtec based and as such have poor performance. Please post a link with the unit you bought. :)
@@zadekeys2194 Hello Zade,
I bought two Genius NIC-U3-02 www.gembird.com/item.aspx?id=9669 with ASIX AX88179 chipset. (Cheap and weak build quality, but it works)
Which type of Realtek chipset did you mentioned? RTL8152 or older? -This one is really weak indeed.
But on the other hand RTL8153 (1gbit) and RTL8156 (2,5gbit) I would go for them.
If you are interested in the supported chipset by the driver:
flings.vmware.com/usb-network-native-driver-for-esxi#requirements
I just got a used SFF OptiPlex and put my 4 port Intel NIC in it. Fits great and works well, and just a bit bigger than these TinyMiniMicro systems.
@@cobalt_3283 Which model Optiplex ? My quad port Gb Intel Nic does not fit in my sff. I think most Sff systems are around 2x the height of these TinyMiniMicro systems.
I'm a big fan of using second-hand hardware for home labbing reasons. I'm still rocking 3 Dell Optiplex 7010's and a PowerEdge T320 in mine. And they are solid!
For low-compute needs laptops can also work well. You get an integrated UPS and local management console. They're probably not as good a choice overall as these micro desktops but if you can get them more easily...
So fun fact, the first server I ever managed was an on-campus UC Santa Barbara Counter-Strike server back in 2000-2001. It ran on a Costco Compaq laptop. During that time California had rolling brownouts and the dorms would lose power but the CS server would still be online.
i love these machines, we use them everywhere where i work
Can't wait for follow-up videos. A dedicated one showcasing the best use-cases in these for a home or small business network would be awesome.
We use HP ProDesk 600 Minis heavily where I work. We even use them for some of our business critical systems that need to be on 24/7 in the server room. They are very reliable and we almost never have issues with them.
I use a Lenovo ThinkCentre M720q as a media center, pretty happy with it.
I recently purchased three Lenovo M900s for a Proxmox cluster with ceph and they work great in my lab environment. Cheap HCI.
This project could be a whole web site by itself.
This project fueled my ambitions to set up a high-availability bare metal Kubernetes cluster up in my home (not because I *need* that, per se, but this is what I want to learn... and I can put it to good use). I'll be using some HP ProDesk machines I got for about $150 a pop for the control nodes and the etcd cluster. But then some of the EliteDesk 705 G4 nodes with Ryzen 5 as workers.
Keen to see more of these!!
Keent to see where this goes man
Looking forward to this. I still have Acer Revo 3700 machines in my hone lab, Atom 525 processors and a whopping 4gb. Ran an entire Windows Server 2008 on 4 of these. Mostly they are idle right now.
I have been using the Lenovo Tiny M72e and M92p for small home server applications for a while. I am currently using Tiny M73's with I7-4285T cpu's for media server applications. The beauty of the Tiny models are that you can swap CPU's for faster better CPU's such as the I7 T model cpus. I currently have a M910q and waiting for a cheap I7-7700T to appear cheap (or try one of the ES on ebay)
Man, you gave me to thing about something... That WSL stealth hosting approach may actually work for me 🤣🤣🤣🤣 Thanks!
I have the Lenovo ThinkCentre M720q / M920q Tiny at work and the are build like a tank :)
Considering buying a Lenovo P340 Tiny recently. Looking forward to the follow up videos.
Using a Fujitsu Esprimo with a Pentium G645 and a dual Intel server-nic I bought for about $99 years ago as pfsense-box, it's been rock solid for like 4 years now.
Regarding power, you might be able to find a sufficiently powerful PoE splitter at the correct voltage, which greatly reduces clutter when clustering those as those are now all powered from a central PoE switch.
This is awesome. I've been using a lenovo thinkcenter SFF with VMWare ESXi and running services for home off it. I do have some issues (some timer overflows and resource overprovisioning fails) and i'm also considering an upgrade or a second one. I can't wait for this series.. i hope to see either clustering or high availability here as well on the cheap / free home licensing etc.
Have you experienced any issues you found to be due to hardware compatibility?
I use mine as a backup server. Pulls files via rsync, then deals with the heavy lifting of compressing, encrypting and uploading to the cloud.
When needed it'll also host servers for me and a friend or two.
In future it may end up doing more, not sure what yet.
I use an HP G4 400 Mini for Plex! It has a 10k Passmark CPU and 32GB RAM in a RAMDRIVE for any transcoding (rarely needed) and a 1TB NVMe for Metadata. Storage and Backup are on two DAS units from QNAP.
That's stupid unless you managed to get it very cheap and have space limitations. It's like $767 USD on Amazon with less memory... You can get a Ryzen 2600x for $80 and related parts and hit 14k passmark for less money in traditional PC case.
@@xephael3485 Why would you say something is stupid without knowing why I chose that setup? I already had the G4 laying around not being used. I ran the numbers and QuickSync on the Intel i5-8500t is more than enough for my needs. Just because it isn't "the cheapest" way if I had to buy it in addition to the drives and enclosures doesn't mean it wasn't the best way to go for someone who already had it.
What connection do you use to the DAS?
@@connorazzarello5514 They are using USB 3.1 but I've since built out a server in a 12 bay Rosewil case using two PERC H200 LBA controllers so I can take advantage of individual drive health on the unRAID OS. In fact, the server has been built for 3 months and I have not made the time to move my library to the new server yet.
Patrick cornering the market on minimicro servers awesome!
Before they became so small I got 3x hp 6300 sff. Amazing things 3y ago, remote manageable in canonical MAAS icw k8s. Been running nextcloud to even my ubiquiti unifi controller for years now.
Love this idea. I have recently been intrigued by the new USB booting feature on the Pi 4. I've been a proponent of low power draw, low noise units for a number of years.
Windows is not an issue for me, as I do mostly Linux and BSD servers and projects.
That sounds great! Can you also do a video on clustered storage like cepth or gluster? When you have a cluster of these you don't really want to go without any kind of resilience, but how am I telling 😂
Agree. Clustered storage is super interesting. Though, a common option that Kubernetes supports is one good NAS / Database machine and everything just runs over the network.
Looks like a good way to build a VMWare cluster lab and learn VSphere.
Check the NIC model, though, because ESX doesn’t like Realtek and the Dell Micro’s usually have them.
Those proliant microservers are nice! I currently run a Pi 4 with ProxMox but, I'm needing more CPU power. (And as quiet as it can get it). Great video!
very interesting project man! go ahead
Your opinion about RUclips falling under professional productivity differs substantially from my employer’s
I use one of the Dell's running XCP-ng and a DC vm. Love this segment of machines.
Why do you need little dinky PCs when you have XCP-NG? It's a ton of servers at once...
xephael when what you want to do requires a cluster of separate machines.
@@williamp6800 what requires a cluster of stupid little hardware machines when you can provision and have a cluster virtually on a single or multiple servers with far more performance?
xephael apparently nothing that would of any interest to you.
@@williamp6800 try to use a fully formed sentence that makes a semblance of sense.
My company was selling old Lenovo Thinkcentere Tiny PC's for $30 It didn't come with windows, but I have it and I prefer linux or OS X anyhow. I have one M73 (i3 slow) and one M93 that I slapped 16gb of ram in and a 512gig HD and made it a hackintosh and it's been awesome and stable since March for me.
Definitely agree with buying second hard refurbished IT Recycled hardware! With c-19 there will be so many pointless desktop devices in offices ready to go to IT Recycling as folks will be supplied laptops, the Lenovo/Dell/HP Office machines have nice i5 and i7 processors (4c8thread) and have Intel VT allowing you to use VMWare etc. Remote KVM/vPro is awesome too!
I looked at doing this but I ended up buying a stack of Lenovo x230 laptops i5 and i7and put SSDs in. Worked out being pretty cost effective.
I've been looking to get one of those as Patrick said that they're really efficient as well as give you more power to do more stuff than raspberry pi.
But costing of those in India is so high, that I'm still looking to get something better for as much low as possible.
Pis being unobtanium what led me to this channel. At the price the scalpers are asking these are a great alternative. Planning on using something like this to run Octoprint for a small print farm.
Welcome Nik
What i need for my firewall, is a tiny computer, 3 gigabit ports, an AES and ECC CPU/RAM. The Lenovo M90nano came close, but then they dropped that second LAN port, though i'd prefer an AMD CPU like a 45/4600u with ECC
I should note than my (most) of the tiny computer's i've dealth with, will support 15mm drives
I use a Lenovo M73p as a home server. These 4th gen machine can take Xeons and they are dirt cheap on aliexpress or ebay. Also, on my machine the mobo has its own power limitation, so a regular 87w Xeon chip works fine.
Did you modify your motherboard on your M73p in order to allow it to run the 87 Watt TDP Intel Xeon CPUs?
Plus 1 for the stealth lab tip!
Have you run WSL2 for real? I do, it is still a rough ride. As a kid I would get rid of that CPU and memory eating monster. Even after tuning the memory compacting is aggressive ands wastes CPU.
@@hcjkruse No I have not ever used WSL. I've been running full fat Linux for almost a decade (desktop and server) and only use Windows when I have to.
@@camerontgore My work laptop is Windows. Running Linux and SmartOS at home. Using WSL2 to get things done: Using Ansible , Molecule and Docker
@@hcjkruse Nice! 👍
I have a couple of the Dell's that I use in my home lab and I agree for small setups, they are awesome and reliable. Just put a small boot drive in there for VMWare and use a NAS for the VM's. The only issue I've run into is with VMWare ESXi 7.0. It no longer lets you inject drivers and I'm stuck running 6.7. So keep in mind before buying, make sure the hardware is compatible for what you plan on running on these things. Because of the form factor, you can't always stick an alternate NIC or something else in there easily.
Instead of ESXi you could try Proxmox or XPNG+Xen Orchestra which are more hardware friendly. Especially proxmox which is basically a Debian.
This is hilarious. I was running. Esxi on an older Dell usff tiny. Loved it. Same issue. Stuck on esxi 6.7. Moved to XCP-NG. Bought second node. XCP-NG pool. Yay!!
Just recently migrated from XCP to promox. Added third Mini PC node.
PROXMOX HA cluster on 3 nodes with Old HP server as shared NFS storage. So much fun. Really liking PROXMOX. Loved XCP. Learned from ESX. What a journey. :)
viv-Home Lab Revolucion!
3 years later, I'm watching this due to the news of the Intel NUC being sunset. You should update this considering the migration from NUCs to other things!
That was just the start of the series. Two weeks ago, we hit this level: ruclips.net/video/HylKpDmwaFA/видео.html
@@ServeTheHomeVideo Thanks for the reply. I guess what I meant was, a comparison between the models, from a “which one works for you” perspective - for someone moving away from NUCs. I didn’t explain myself well in the original comment.
I have 2 4th gen based Minis, a HP Elitedesk 800 and a Lenovo m73p. These still usable as a mini server? The HP has a i7 and the Lenovo has a i5. I seen they can take Xeons so I plan on upgrading to a 4 core/8 thread Xeon soon
I'd love to see a test using them as a Parsec client. I have a system using a "Pentium G3250" and using the igpu I had some issues. I know anything somewhat current gen would be better but I cant find anything "official". I love the series and am currently bing watching it!
I love and like lab
Can you do a video on getting a 3+ node cluster going with proxmox setup??
Interesting for at home render farm
So cool! Looking at those Proliants and wanting to ask HP about getting an Integrity in a TinyMiniMicro form factor. (never gonna happen? Oh thanks, Intel.)
I've been out of the (home) server world for a number of years now. Used to run a linux server providing web, file, and media serving capabilities. For a home user, what advantages does kubernetes provide? How does this vary if you're running just 1 server vs a couple of servers? Thanks!
I agree that RPi 4 isn't really good for servers/nodes for the price/performance but I feel like arguing using the kits as the baseline isn't an effective angle.
I think a better use case to look at is RPi 4 with PoE hat, no case or 3D printed/homemade case/frame, SD cards usually bought outside of kits are cheaper.
I still think for the performance you get a RPi 4 cluster loses but it would've been a stronger argument.
Also things like SD card lifetime will get less relevant when network booting gets implemented.
Overall I still agree with you that RPi 4 isn't the best for this sort of thing
What about a LattePanda Alpha? You can use a M.2 NVMe SSD with this!
@@markarca6360 Just looked at it and it's more expensive than the PCs that they are using in the video, plus they are dual core.
I think they would be worse still than the RPi except maybe in the performance density.
It's hard to beat the value of used hardware with brand new hardware, it's just that simple. These ultra compact computers are just so robust and well built that, frankly, they should easily have a longer lifespan than the more DIY oriented stuff like RasPis and the like, so buying used in this instance isn't much of a tradeoff for a DIYer. Nevermind the performance difference of course, but there's also a wealth of used PC hardware that you can even upgrade these with on the cheap. And, if you're the type that would 3D print your own RasPi case, imagine doing that for one of these to, say, fit a larger CPU cooler with a bigger, lower RPM(quieter) fan.
Hard to argue against, especially when they're plug-n-play, even including an OS license(should you seek that out). And the manufacturer support between the two options is on opposing extremes of the spectrum too; if it wasn't self evident, one is hobbyist level, the other is professional level, lol.
I know it's the 'cool kid' thing to do to crap on x86 & Windows these days - but the bang for the buck is just on a different level from everything else, they'll always be a viable option for numerous use cases regardless of what happens on the bleeding edge of the market. If you're a gearhead, it's the small block Chevy of the computer world; it may not be the best at anything, but it works pretty damn good for just about anything, and your grandmother could build one blindfolded for the cost of couch cushion change and some pocket lint...
They're also available in i7 versions as well. I find they're great little machines for projects like HTPCs, retro gaming machines etc. I haven't set one up as a server yet, but thats on the cards. And while Windows 10/11 Pro might be a good option for some, some of the machines I tend to use Linux on as it runs better with the older processors.
I considered a couple of these for PiHoles... But ended up with a CanaKit RPi.
Same here
Just mix them with the Pi 4 8Gb please ;-) If you do cluster in K8S a X86-64 and ARM64 mix would be nice for home and improve that use case and development.
use zentyal as the active directory server, my 2 win10 Pro are registered in that directory,
i can logon on the workstation pc using any account made on the server.
server has a web gui for remote management,
can even use domain policies managed from any workstation.
It’s too bad you couldn’t include something like the SuperMicro’s SYS-200 or SYS-300 (maybe price was a reason?). I’ve been using Intel NUC Skull Canyons for NVMeOF testing, but unfortunately they don’t support IPMI. Recently picked up the SYS-300, and they’ve been great for Ironic/MaaS testing for bare metal provisioning. Great video btw; very helpful!
So what are you you using for shared persistent storage across all your nodes? I'm assuming if you launch a new instance of MYSQL it won't have access to the data file of the instance currently running.
My first home cluster was built on fanless mini-itx computers for the express purpose of ripping all of my Audio CDs. I completed the task 3 times because of catastrophic failures [a] accidentally deleted everything [b] disk crash before backups [c] higher bitrates... it was fun as projects go but a waste of time as we now have streaming audio... I've also built multiple docker-swarm and kubernetes clusters. While I like the rancher product failures happen and recovery is never pleasant. In the last few years I've been running multiple VMware ESXi servers without vCenter and it requires an expensive API license in order to automate/orchestrate services etc. (now investigating proxmox)...
Home clusters are fun but once you start to do REAL work on them you need REAL hardware and your raspberry pi comments are spot on... that first cluster will be replaced with something more expensive so do not waste the cash on the first cluster. Go right to the second one. Now the question is what to get? That's a really hard problem because you do not want to be manually distributing VMs or containers. Also you do not want too much free space or idle time... Scaling down an enterprise operation is a serious challenge and this has costs... it's the sort of thing that has me going cloud. VPS are cheap and can be turned off to save money.
on big question to me is storage.
newer models of those 1l category have 2 m.2 and multiple usb 3.x 10gb/s ports.
having multiple nodes clustered with each holding:
1* 120-500gb m.2 for os and vms (thinking about hypervisors like proxmox)
1* 1tb m.2 for fast storage
1-4* 10tb+ slow hdds for large data using usb 3 10gb ports.
clustering those fast and slow storages into a pool using glusterfs or ceph or any other solution.
so i want to ask:
instead of connecting the slower and larger drives via usb 3 instead of sata (as there are no internal ports or even space for those drives)...
does this become a bigger issue?
in terms of speed?
in terms of fail-safety (like a raid but in a hci-approach)?
this is not meant for a production environment; just a homelab with the wish not to waste speed and reliability on poorly made decisions ;)
(a reason i stopped thinking about raspberries)
btw: i am curious about using lenovo thinkstations or such, as they have a additional pcie port, to add a 10gb/s interface, in order to maintain fast speed across the storages.
thanks for any input in advance :)
That Kubernetes Example sounds interesting, but i somehow fail to imagine how that would work with a client placed @ your parents house for example. Does that Kubernetes Node there connect to a for example selfhosted instance somewhere in the net ? How would you use some of the ressources @ the client so that it actually makes sense to run something there ? What software would i want to run there ?
I'm just wondering, where do you typically source these things from on the second hand market? Ebay?
The big OEMs have outlets for refurbs/ overstock. Sometimes Craigslist or recyclers I know.
I have been thinking along the same lines. Got some HP Prodesk 400 G4, I am amazed how compact and quiet these are! Planning to run 3 in a Proxmox cluster. Question: my boxes had win10pro install any way these can be ported ? Clone it perhaps into a small nvme or just extracting the key is enough? Like the sound of WSL2 on windows, experimenting with this tonight 😁. Looking forward to the series.
for the HP Prodesk 400 G4 there is a publicly available download for a recovery iso.
when you install that iso, you do NOT need to enter a key. On first Startup and Internetconnection it gets activated
@@rootgremlin2746 You don't need special HP ISOs. The standard image from Microsoft works as well.
@eDoc2020 Not the one from your MSDN Download, and you would need to enter the HP-OEM Key. Which my Prodesk did not have. So the easyest way was the USB-Install-Stick creator on the HP Downloadsite
I love little machines. I am one of those crazy people who ran out and bought an i7 mac mini back in 2012 to run esxi.
I wonder about the price/performance of the various J4105/J4115 machines (Seeed Odyssey, Odroid-H2+, Chuwi LarkBox, etc.) popping up lately compared to EOL surplus business minis. Particularly WRT clustering.
For me more insteresting thing is comparison between Jxxx/Nxxx cpus with "T" and "TE" versions of normal CPUs (35 and 25W TDP). Because it seems that in terms of small homelab usage where most of the time machines run in IDLE, there is very small (if any) difference in power usage between power-saving cpus like Atom and normal cpus . Some even claim than normal CPUs are more power saving because even though they take more power under load, they finish their tasks far more faster than those "power-savy" cpus.
These things are insanely cheap on ebay. Pair with NFS/CIFS NAS storage or create a cluster out of these...
This is my next step!
Not for long after this video makes its rounds.
... or both -> CEPH, GlusterFS, DRBD
@@Garageland16 Yeah, exactly, they _were_ cheap on ebay... ugh
I am wanting to do multiple nodes to practice vmware's cluster stuff like vmotion and vsan.
I would really love to see from you some some possible workarrounds of lack of AMT. IMO AMT thing is main thing whether it is worth get them or not. YOU HAVE TO HAVE KVM to mangage cluster of them efficiently. Sadly in my country (Poland) this feature most of the time is disabled. And as you may know, once you disable it you cant (?) enable it back. Ive seen stories about successfull reflashes with Intel's Flash Image Tool kit which re-enables AMT. But i havent seen any tutorial to even confirm its possible. I also have seen some DIY KVMs made from RPI0 + chinese clones of CamLink grabbers. Please, take a look on this, you would help for a lot of people. (I just subcribed your channel)
Pi4 also supports USB boot without the need for a SD Card, (not even for the firmware anymore) So slap it in a nice box with a USB3 SSD and it's not that bad really. using one as an LXD Host
Hi Ron. Totally. These just support USB (including newer 3.2) as well as SATA and NVMe SSDs. Again, not saying RPis are bad in any way. These are just marginally more expensive with more expandability. I like to provide options for folks.
I am using an HP Elitedesk 800 G4 Mini (65W) with 32GB RAM, i7-6700 running the free Windows Hyper-V Server 2019. It is amazing for a home hypervisor just by itself!
I want to add, though, that the "whine" that the super small fans (HP 65w model has two, one for cpu and one for the nvme) cause is more annoying to me than the fan noise of the Microserver Gen8 at 13% speed. The 35W model is more bearable but still the small fan has a very annoying whine to it similar to coil whine. Not sure if the similar Lenovo or Dell Optiplexes have that part figured out.
ive wanted to maybe one day get one of themUSFF type systems as a music pc but the elitedesk line g3 and onword cost a decent bit of money still
My home lab is 2 tiny PCs. And a old. HP server as NAS/SAN
I want one of these to try to set up for recording my Security cams. Wonder if these have the power to do so? 2 -3 cams at 1080p IP poe cams
These are great. Curious if they’ve got intel QuickSync on any of the models. To the google!
The Intel ones are 90% of what's out there and they all have the integrated GPU -- discrete graphics is added on some 'workstation' units (Lenovo P320 is one). I think that means SquickSync -- are there ones with GPU that don''t have QuickSync?
I went down a different route and used an Atom based industrial controller. It does basic NAS duties & is a DVR for my security cameras. Your mini-micros are way more powerful, and I will try checking ebay etc. However as my current box usually sits at about 15% utilization, it can stay as it is for a while longer.
It is running Win7, yes bad, however it does not do desktop duties, like browsing & email, so hopefully it flies under the hackers radar. Having to regularly log into a Win10 box to 'settle' endless Microsoft feature updates does not appeal to me in any way, and Linux Wine is too patchy I find, so it's Win7 untill I find a good altenative.
Some have Intel AMT so some sort of remote management ;)
I just was at my university surplus store and they had I7 ones of these micro PCs for sale for only $100 each
Wow! Sweet!
How well do those machines serves as thin clients? Like for gaming or workstation?... high frame rate/ refresh rates... fast input?
They work OK. These are like a normal Core i3/ i5/ i7 system GPU and CPU, just usually lower clocks. They are probably not something you would want to game directly on though.
Any ideas how to make m93p withi5 fanless or reduce the fan noise (for i3)
What's your view on having a AMD Threadripper 3950X (16C/32T) and create 7 of (2C/4T) virtual machines) (utilizing 64GB of RAM)
I personally think the VM way saves more time and trouble because these tiny nodes typically have their own wearable parts. (Fan could die, motherboard could die)
I also think it is more cost effective to have 1 big server with 6 nodes VM
If you do the math, the 1 big server approach is actually cheaper than buying 7 small nodes.
That was part of our Of BBQ and Virtualization video. For some sense, we have been buying 16GB 4C/ 4T nodes with 256GB of storage for under $250 each with Intel AMT/ vPro for remote power on/ off. So 16 cores, 64GB, 1TB for about $1000 with remote management and 4x Windows 10 Pro licenses as well.
On the failure bit, that is very interesting. You are trading a higher probability of failure (more components and systems to fail) with a much smaller radius (one small machine versus the entire big machine.) You effectively get a higher chance of 1/4 of your cluster failing, but a significantly lower chance of your cluster going offline.
I have the ASRock Rack X570 mITX platform next to me, so this is something Will will likely talk about in his STH main site review of that.
I have tried running an Intell NUK 2 times, by two different venders. Nether of the Nuks lasted longer than 18 months. I love the idea of a cluster of small systems, thanks for this video series.
Trying to figure out the use of a home computer cluster made up of ordinary PCs
Aside from high availability, containerization and distributed databases and forgetting the power inefficiencies, is such a cluster useful in solving some sort of problem at home, which requires the combined computing power of more than one node at the same time?
I am aware of a single application of this sort and that is getting two or more NVIDIA GPUs across several computers to simultaneously train a given machine learning model.
Thank you.
How is the power supply situation for these micro devices? External bricks? And if so, how big are they? Size and wattage wise.
We covered this a bit in the video/ article but they are usually 65W power bricks that are shared with the vendor's notebooks. Fairly small and very easy to source.
Hopefully you have better luck with those HP Elitedesks than I have at my work in the long term. They tend to suffer a lot of issues in my (limited) experience with them when it comes to windows updates. It became so much of a headache dealing with a failed windows updates or an overheating unit that I stopped purchasing them.
Still have not seen the update but that may be because of running them for only a few weeks. We did have one that made a cameo in this video that had a ton of dust in it when it arrived which made it run louder.
@@ServeTheHomeVideo Based on what I've seen at work, if you ever see one of them with a blinking red light and or beeping and there's no picture on the screen it probably overheated itself. Of the 7 or so I deployed at work, 2 of them became problematic with overheating.
However, to put all this is perspective, I work in a machine shop with all sorts of contaminants in the air that has a history of just destroying hardware sometimes. Sorry for not mentioning that with my first comment, I'm so conditioned to it at this point I sometimes forget our environment is very much so not ideal.
@@vitelliu5 not running windows on theses "servers" solved a lot of problems ;) (not overheating obviously tho)
So these are basically small servers, right?
Do any of these 1 liter PCs come capable of using ECC memory? I'd like to employ multiple promox nodes in my home lab, but care about the data that is on the system.
They do not. The vendors do not validate ECC on them so it is not something I would rely on
@@ServeTheHomeVideo after some more digging around I found that HP Z2 Mini G3 xeon variants will support ECC. You did a video on the G4. Slightly above the 1L size, but doing some more videos on these might be good people who want the comfort of ECC like I do.
What are the nodes for?
How many nodes should I get? I've never done before. I was going to do 8. Is that too many?
now that you have all of these pc what do yoy do with them?
Regarding Raspberry Pis, are you aware of the Turing Pi? It's currently out of stock but might be worth taking a look at. It enables you to have up to 7 RPi compute modules as a cluster, run Kubernetes on them etc.
Totally. Could not get one though. These are much bigger memory footprint/ storage nodes.
True the Turing pi's awesome but those old compute modules are too weak for anything more than testing. Check out Jeff Geerlings benchmarks on it m.ruclips.net/video/IoMxpndlDWI/видео.html For that amount of money anything sadly is of better value! At least until they release a RPI4 compute module that is.
Do any of these TMM nodes have ecc Support?
Most we have are Core i5/ i7 based so no ECC support on the chips.
@@ServeTheHomeVideoThought so... in a home lab you do not need this but if you want to go professional with this its a feature i would not wantbto miss...
Thx for the answer and as always awesome channel!
Newb question, can you chain them together to power one vm? I'm thinking of this for 3d rendering
Matt - not really. If you have a job that can split into multiple small VMs, then that will work. If you need a big VM, you cannot fit them onto smaller machines like this. Check out our "Of BBQ and Virtualization" piece from a few weeks ago that illustrates this using BBQ ruclips.net/video/dgov7184za0/видео.html or web version: www.servethehome.com/of-bbq-and-virtualization-large-nodes/
reminded of older clustering tech and thinking about using that with these.
WSL2 does not need windows 10 Pro. Home wil do. Pro is nice for full Hyper-V and remote desktop.
Windows 10 Pro licenses are sometimes auctioned for EU 10-20.
This video is interesting pricewise compared to my mini-itx cluster stuff. It is harder to get or assemble reasonable quality nodes using mini-itx in Europe.
Hi could you possible tell me what those dell mini pc’s their name.
I was thinking of getting two of them and upgrading the processors so i can upload and do work on them with both handling half of the load?
Optiplex sff
Is there an open source hypervisor capable of running VMs (or at least containers of some sort) that could live atop a dozen of such budget machines, strung together with Infinband? Programming a hypervisor that would take into account a number of quasi-remote CPUs sounds horrific but I'm still excited by the idea of duct-taping together a dozen identical desktops and calling it a supercomputer.
You would likely want a larger unit for a PCIe x8 or x16 slot for Infiniband.
@@ServeTheHomeVideo Were that hardware requirement somehow met, do you know of a software solution, capable of creating (from a number of machines) a hypervisor, which would be more than simply a sum of it's parts? There are a number of clustering solutions (Proxmox, oVirt etc.) capable of providing failover clustering; Is there one which could join physical machines at the hip? By the way: The videos you guys put out are great, often time they brighten my day. Thank you for that. EDIT: A little bit more research suggests that what I'm after is "software defined server" technology, the main proponent of which is TidalScale. I'm not hopeful about FOSS alternatives.
Raspberry Pi 4 Cluster !!!
What generation of CPUs are these generally using?
Are there any particular features (such as VTd) that are a must?
:)
Typically we are focusing on i5-6500T/ AMD A6 and newer CPUs. If you just need something more basic the i5-4000 generations can be had for well under $200
ServeTheHomeVideo Thanks for the reply! Much appreciated :)
Great assessment of Pis vs Micro nodes!! The Pis are totally overrated for the price. Only pro is power consumption. Do the Mini nodes really use more power than Rpi?
"And thats when I just started buying nodes" yeah I have that problem too
Hi thank you so much for this project and sharing. Can I ask maybe for some recommendation, which manufacturer would you recommend ? Lenovo|Dell|HP ? is the i5 sufficient for a small home kubernetes setup ir should I rather go with i7 which has HT ? thank you
I think the i5-8500T and newer are greeat. Until the latest generation I think Lenovo has the best hardware, then HP, then Dell.
@@ServeTheHomeVideo Hi those models are pretty expensive. The current refurbished have either i5 6500t or i7 4765T they are like 200euro, I was searching for the model with the i5 you mentioned and they are over 700, correction found some refurbished below 500 still not better then the i7
@@ServeTheHomeVideo actually I found the EliteDesk 800 G4 with the 8500T with decent discount so I will grab it the as I can much cheaper get an DDR4 32GB memory kit as DDR3 16GB, thank you a lot guys, appreciate the project you do and wish U the best of luck. Martin
@@ServeTheHomeVideo Hi guys thank you a lot for the awesome insight. I focused on finding an i5-8500T cause of the 6core and this also support vPro. In the end what was the breaking point was that there was a cap for the old i5/i7 an 16GB DDR3 limit memory. the newer models have like 64GB. I have currently bought an 512 SSD + 16GB DDR4 on Otiplex 5060 Micro SSF. and love this device. Way way better pick than an RP paired with Proxmox, awesome. Cheers guys
My T620 idles at 210 W....