Great video and nice list. Broadcom is destroying VMWare and we have been moving many clients over to XCP-ng. I love FreshRSS what plugins are you using with it?
Thank you @lawrencesystems! Humbled you enjoyed the video. Definitely think there could be books written about what Broadcom is doing "How to destroy a company in 6 months". I need to spin up a fresh XCP-ng instance again as I know they have made many improvements over the past few months. I am looking forward to their host client they are working on. I don't have too many plugins installed, but am using "auto refresh, colorful list, share by email, and three panes view (flag this on from time to time). Let me know if there are others you would recommend. Thanks Tom!
would love to see a video about power management, mainly ups contingency during blackouts and organized automatic shutdowns across devices. I've found this to be particularly challenging with a mix of devices.
I don't like running ceph on something like an MSO1 with limited networking even though it is 20 GB, unless you have maybe 6 Plus nodes. So I use replication in proxmox on a pretty dang short schedule so if a note goes down the workloads will stand up on the other node and I will have lost maybe a minute or two but that's it. Plus migrating is just as fast but read rights are blazing off of local storage
I love all these videos at the end of the year when all my favorite RUclipsrs go through the hardware and software they're running in their labs so I can get ideas for all the irresponsible purchases I'll want to make heading into the new year...
Thanks for the share, Brandon! Love these end of the year walkthrough because they help me create my project list for next year. Do you have any videos or more info on the weather station you mentioned in the video?
Still in virtualization here.... not thinking in switching to docker swarm/kubernetes in short time, but this phpIpam I will install ASAP... great video!
It's not a huge undertaking or a switch. They compliment each other. It's just about running a container in proxmox or whatever hypervisor with docker on it and portainer running on docker to make template browsing and deployment easier so you can run things that easily install in docker in there. Especially things available in that form. If you want to learn docker, then don't use portainer except as a backup. Do it all in the cli on a VM. Kubernetes I would definitely learn and deploy the control and one worker node per hypervisor node. And learn all about it. It's fun. Also don't so on your hypervisor. I test in docker then I move to kubernetes when I'm sticking with a solution that I built or that somebody else built
MINI pcs are the best and only way to do a home lab.... did it along time ago i would never touch a full rack server again with kubernetes and proxmox who needs anything else
Bro that's NIIIIIIICE!!! Quick question, what rack is that? More importantly, would you buy that one again? And, would you recommend that one or another for a new HL builder?! Thanks for walking us through it!
I have just started looking at microk8s, would you be willing to do a tutorial on how you created a cluster, got your ingress/storage/dashboard working ?
nice video love it! one question: how do you backup a database container with duplicati? e.g. with "normal" container you can create a .tar of data files but with database you can't because you risk having a corrupted database and for now the only way i found is stop the container, create a .tar and restart the container (using cli)
what kind of bandwidth are you getting from your ceph setup? - I attempted a 5 node proxmox cluster on top of a few dell sff mini pc's with 2tb nvme drives running on each with the stock version of ceph baked into proxmox and I was extremely disappointed with performance. I tested by spinning up a windows vm in proxmox with ceph as the storage and got 748/134mb/s for sequential workloads and 10.48/.09mb/s for rnd4k which I found to be unusable - and I suspect that will get worse when running more than a few vms that were storage bandwidth dependent. I couldn't imagine running anything that needs a large database on something like that. I'm thinking about scuttling my ceph setup (homelab testing), and trying out linstor as i've heard it's more performant for small clusters.
I am having performance issues too... my zabbix keeps alerting "read/write responses too high"... I'm already thinking in abandon ceph and build a openmediavault NAS instead... using one of the machine nodes... Here i have a 3 node cluster....
@@frdurso does OMV support some kind of redundant clustered mode? I've never touched OMV. I *do* have a (mostly spinning rust) truenas scale setup that I use for primary archival storage, but because it's hard drives it isn't as fast as I would want my VM storage to be. also a single nas setup becomes a single point of failure. if that node goes down (because of a problem or for basic maintenance like updates or whatever), you lose your entire cluster. the reason I wanted a clustered HA setup is to minimize downtime as much as possible. I don't want to offline my whole cluster every time I get a new version of truenas scale.
@Cpgeekorg I do think Ceph is one of those solutions that you need to throw a lot of hardware and network at. But to be fair that seems to be the case with most HCI solutions. There is a LOT of write amplification with Ceph in just a basic replication configuration.
@faisaldarbar I usually have a core set of hardware that I treat as "production" and then have the hardware where I stand up new ideas or earmark as new production equipment. Then it is just a matter of migrating over workloads and resources.
If mini pc’s got IPMI that would be the day I’d switch. I’m talking about built in no pie kvm crap clutter. As far as ESXi I’ll keep using it until Broadcom messes with the activation process still works right now. It’s just more polished than Proxmox.
@Cooper3312000 I definitely agree on VMware and that is what has kept me here for now. Things are getting much better on the other solutions, but still not quite there yet.
Great video and nice list.
Broadcom is destroying VMWare and we have been moving many clients over to XCP-ng. I love FreshRSS what plugins are you using with it?
Why not proxmox ?
@@basdfgwe XCP-ng is better suited for scalability in business environments
Thank you @lawrencesystems! Humbled you enjoyed the video. Definitely think there could be books written about what Broadcom is doing "How to destroy a company in 6 months". I need to spin up a fresh XCP-ng instance again as I know they have made many improvements over the past few months. I am looking forward to their host client they are working on. I don't have too many plugins installed, but am using "auto refresh, colorful list, share by email, and three panes view (flag this on from time to time). Let me know if there are others you would recommend. Thanks Tom!
would love to see a video about power management, mainly ups contingency during blackouts and organized automatic shutdowns across devices. I've found this to be particularly challenging with a mix of devices.
You checked out nuts/networkupstools
I don't like running ceph on something like an MSO1 with limited networking even though it is 20 GB, unless you have maybe 6 Plus nodes. So I use replication in proxmox on a pretty dang short schedule so if a note goes down the workloads will stand up on the other node and I will have lost maybe a minute or two but that's it. Plus migrating is just as fast but read rights are blazing off of local storage
Thanks for sharing some info and details. Awesome. Have a great day
I love all these videos at the end of the year when all my favorite RUclipsrs go through the hardware and software they're running in their labs so I can get ideas for all the irresponsible purchases I'll want to make heading into the new year...
Great video and some great ideas to deploy. ty
Did you say Twinax? Is there a System/36 in that rack?
Thanks for the share, Brandon! Love these end of the year walkthrough because they help me create my project list for next year. Do you have any videos or more info on the weather station you mentioned in the video?
Still in virtualization here.... not thinking in switching to docker swarm/kubernetes in short time, but this phpIpam I will install ASAP... great video!
It's not a huge undertaking or a switch. They compliment each other.
It's just about running a container in proxmox or whatever hypervisor with docker on it and portainer running on docker to make template browsing and deployment easier so you can run things that easily install in docker in there. Especially things available in that form.
If you want to learn docker, then don't use portainer except as a backup. Do it all in the cli on a VM.
Kubernetes I would definitely learn and deploy the control and one worker node per hypervisor node.
And learn all about it. It's fun. Also don't so on your hypervisor.
I test in docker then I move to kubernetes when I'm sticking with a solution that I built or that somebody else built
Thank you for sharing.
MINI pcs are the best and only way to do a home lab.... did it along time ago i would never touch a full rack server again with kubernetes and proxmox who needs anything else
Bro that's NIIIIIIICE!!!
Quick question, what rack is that?
More importantly, would you buy that one again? And, would you recommend that one or another for a new HL builder?!
Thanks for walking us through it!
@omarlittle5802 it is the Sysracks 27U rack. I definitely love Sysracks equipment and would recommend them.
can you make a video on how did you configure Vmware esxi and Vcenter with the featers you mentioned?
I have just started looking at microk8s, would you be willing to do a tutorial on how you created a cluster, got your ingress/storage/dashboard working ?
@vector1one definitely have this one on the books for 2025 so stay tuned!
nice video love it! one question: how do you backup a database container with duplicati? e.g. with "normal" container you can create a .tar of data files but with database you can't because you risk having a corrupted database and for now the only way i found is stop the container, create a .tar and restart the container (using cli)
@lele-tz1uz I think this would still be the safest way with DB containers to make sure there is no data corruption.
what kind of bandwidth are you getting from your ceph setup? - I attempted a 5 node proxmox cluster on top of a few dell sff mini pc's with 2tb nvme drives running on each with the stock version of ceph baked into proxmox and I was extremely disappointed with performance. I tested by spinning up a windows vm in proxmox with ceph as the storage and got 748/134mb/s for sequential workloads and 10.48/.09mb/s for rnd4k which I found to be unusable - and I suspect that will get worse when running more than a few vms that were storage bandwidth dependent. I couldn't imagine running anything that needs a large database on something like that. I'm thinking about scuttling my ceph setup (homelab testing), and trying out linstor as i've heard it's more performant for small clusters.
I am having performance issues too... my zabbix keeps alerting "read/write responses too high"... I'm already thinking in abandon ceph and build a openmediavault NAS instead... using one of the machine nodes... Here i have a 3 node cluster....
@@frdurso does OMV support some kind of redundant clustered mode? I've never touched OMV. I *do* have a (mostly spinning rust) truenas scale setup that I use for primary archival storage, but because it's hard drives it isn't as fast as I would want my VM storage to be. also a single nas setup becomes a single point of failure. if that node goes down (because of a problem or for basic maintenance like updates or whatever), you lose your entire cluster. the reason I wanted a clustered HA setup is to minimize downtime as much as possible. I don't want to offline my whole cluster every time I get a new version of truenas scale.
@Cpgeekorg I do think Ceph is one of those solutions that you need to throw a lot of hardware and network at. But to be fair that seems to be the case with most HCI solutions. There is a LOT of write amplification with Ceph in just a basic replication configuration.
How would a beginner integrate all that hardware and software without needing to pull down everything and building it over and over again?
@faisaldarbar I usually have a core set of hardware that I treat as "production" and then have the hardware where I stand up new ideas or earmark as new production equipment. Then it is just a matter of migrating over workloads and resources.
If mini pc’s got IPMI that would be the day I’d switch. I’m talking about built in no pie kvm crap clutter. As far as ESXi I’ll keep using it until Broadcom messes with the activation process still works right now. It’s just more polished than Proxmox.
@Cooper3312000 I definitely agree on VMware and that is what has kept me here for now. Things are getting much better on the other solutions, but still not quite there yet.
Tk U for sharing. I am interested in learning Promox on HP mini desktops. Currently using VMWare Workstation Pro for my VMs.
I like the content, I just feel like I'm being read to and not spoken to. Please find a way to just talk to the camera and not read your notes to us.