I want to see some content on the scale side so rather than the overkill server as a NAS and virtualization stack in one box. I want to see lots of small to medium size systems “glued” together as a coherent platform
In theory it should be able to handle a failed host not just individual disks in a pool. Seeing it in practice is something I'm really looking forward to.
I run TrueNAS Core in a VM under Proxmox and just passthrough my HBA. Seems like TrueNAS Scale could easily replace Proxmox and let me use the same storage pool directly for other VMs. Excited to give it a try when it's ready.
I've been running SCALE for 6 months now and it's rock solid stable with excellent performance. The SMB shares are really easy to work with and the Plex hardware transcoding is much easier to get going on SCALE than Core.
2 года назад
Yeah, I can confirm that. I also have a couple of storage disks for SMB shares, running VMs and apps like Plex/Docker on it. I'm using SCALE for 4 months now and I really appreciate it. It makes so easy to share, deploy and virtualize VMs from the same GUI. So excited to see the final release next February.
It would be really nice to see some testing of iWARP vs RoCE for the RDMA testing as the whole DCB overhead of RoCE is off putting for home/small-business labs.
I always watched you when you did news on that not to be mentioned channel and you peered through your two monitors to the camera before I was a teen. I've built a few NASs and heaps of PCs now and I'm super excited to see you geek out and push this new platform. That old server stuff is cheap but a few thousand dollars is a little steep (especially as a student's hobby).
I've been running SCALE since the alpha and I have no major complaints. I'm waiting for the stable release before I migrate my synology's data over but for now I've got pihole, plex, jellyfin, multiple valheim (game) servers, minecraft (game server), and transmission (torrents) running via containers. I wanted the highest core count lowest power usage system so I've got: Ryzen 9 5900 12 Core @ 45W TDP via "Eco mode" Asrock Rack x570 mATX motherboard - Future 10GBe expansion 64GB 3200Mhz ECC (i really want another kit before I add spinning rust storage) NVME only pool for now.
You run my dream homeserver setup =D. I like especially the Asrock AM4 Serverplattform, with the KVM built-in. Do you have the 10 Gbe network version of that board? Im surprised, that a Ryzen 3000 Series or 5000 Series could run with such low powerconsumption. Or does it? I assume it from your Eco Mode.
@@MrUltranobody Yes I have the version with 2X 10GBe, I just don't have anything to use those NICs with yet. And yes, the Ryzen chips support ECO mode. 65W TDP Chips (like the 5900) -> 45W 105W TDP Chips (5900X/5950X) -> 65W
Yeah, I know that feature of Eco Mode from the Ryzen 5000. But how much is your power consumption in Idle? By the way, you can get 10 GiB Switches for around 100 Bucks.
Love it! I am a COMPLETE NOOB and having freenas running for the last 5 years.... Brings such happyness having ZERO issues with it... Well, it BURNS USB memory sticks if u try to zfs them... but its all good. Hahaha Keep up the good vid's.
that was one of the friendliest introductions to a new project i've ever seen. especially the part in keeping the support work away from the original developers of truenas (core), i hope that is as appreciated by them as it is by me. i've started tinkering with scale in a vm (on my core machine) just to figure out how everything works. still struggling with how to set up smb shares in a "advanced home network" environment, so no AD or similar to get the windows security identifier... very strange compared to core. but i'll get used to it and i think it'll be amazing as a "one fits all - hyperconvered" infrastructure thing where i can run everything i need on one box. looking forward to that series of you!
You magnificent so and so... I have everything but a CPU cooler for my upcoming truenas scale build and i wanted to do some advanced VM stuff as well (preferably clustering. I was hoping to get proxmox installed along side truenas scale given they both seem to be built on debian iirc). I want to be able to easily pause, move a VM to a different box, and resume a VM while using all the great filesystem management, docker, and smb sharing awesomeness in truenas scale.
I've been running freenas for a long time. I just upgraded to truenas last night. 18 core xeon 128GB of ram and i'm only using it for SMB. I've been having reliability issues with permissions since I started messing with it... I wish it was a bit easier to manage.
Could you do a noobs guide on TrueNAS? You're really calm and easy to listen to, but I feel like Im so out of the loop with some stuff that I would love to learn more from you
I'm running TrueNAS Scale on my old gaming PC, R7 2700x, 32GB of RAM and some IronWolf 4TB HDDs. Works pretty well. This is why you don't throw away old hardware if you can
Really good video - thanks a bunch. TrueNAS SCALE is indeed a cool product, though I think most users will be happy with TrueNAS Core. The main attraction to SCALE are the Docker containers and KVM integration, though the product was initially developed to solve horizontal scaling - a feature that TrueNAS or FreeNAS could never quite deal with in the past. Being able to have native Docker containers I think will make the platform seem more appealing to new users, but for the most stable experience, TrueNAS Core is likely still the tried and true, and should still be the base of the TrueNAS Enterprise products (not necessarily the same build, Core is kinda the beta for Enterprise, but mostly based on the FreeBSD platform with more native ZFS support).
I've been using Scale since early beta, and it's been pretty solid. Using NFS/SMB/iSCSI without any issues. Had some (possibly) self-inflicted routing issues that I've worked around. Not a fan of their wizard/GUI for creating a container, hopefully they add support to dump docker-compose files similar to portainer.
The appliance nature of TrueNAS makes it really easy for my client to manage users. I set it up for them in a VM on a DRBD partition, 2 server solution. Ridiculously cheap, and redundancy beyond what some of the large enterprises i work for can manage. I am slightly worried about the recovery capability of ZFS vs EXT4 in the event of a drive failure & DRBD failover.
Along the lines of your drives saturating the memory bus, can you use something like Intel MBM to report memory bandwidth usage? Note that MBM didn't work on certain versions of Intel CPUs (I know right!?).
I use a DAS as I need fast local high capacity storage. My Switch doesn't have anything faster than Gigabit speeds, and anything faster was prohibitively expensive when I made the purchase. I ended up with a QNAP TR-004 because it has a 5Gb USB 3 connection. Faster than my network, but future proof so when I upgrade and get a faster switch and a NAS I can connect it as local storage expansion. (the TR-004 also gives me hardware Raid 5!) I've rarely seen anyone talk about this advantage.
Awesome! I use a VM running k3s/Rancher to manage all my containers on TrueNAS Scale. I don't like what they've done to the UI for apps and I only run it when I need it. Before, I ran Rancher right on TrueNAS Scale but that caused some problems that grew out of controls after updates
What timing! I'm currently migrating my dual E5 v2 128 GB build to TrueNAS Scale. 4x 2 TB NVMe in RAID Z1. Backups to B2. Everything's great so far. Hourly file history for windows desktops small oopsies, two week ZFS snapshot window for normal oopsies, daily offsite one-ways for big oopsies. zstd-3 and dedupe can run at my line rate (10G). I have yet to play with iSCSI for steam library, but it's on my list. It's so nice to finally have a high performance and reliable local VM host for my software playground wants.
There's a lot of low frequency noise in the sound, I think from bumping into the table or something. Maybe have a look at isolating the mic from the table better? Now I keep thinking someone's knocking on my door :P
running it on mini x+ could not be happier used only for family nextcloud and so far the upgrade process have cause minor issues other services i have running heimdall , home assistant and traefik to which i can't understand at all yet since its more capable than nginx but lacks a gui . just a note in case other users of scale are reading if you will install traefik or similar remember to change your trunas scale ports, at least for me when i try to edit config.php for nextcloud need to do a apt-get update then install every time nano so i can adjust the file. cheers looking forward for next video
I'm super excited about TrueNAS Scale and this video series. Would love to see how to configure ZFS for good performance with spinning rust and an SSD or two.
Will there be any "scale out" content in the future; ie multinode TrueNAS Scale node testing? I can't recall if i've ever seen someone using the scale out feature (Gluster?) in Scale, so that would be interesting, especially failure modes & recovery
Can we stand up a DIY video streaming service with that type of bandwidth? (Kind of like "guerilla RUclips") Or is that more bottle-necked at the network bandwidth? Or, with that much data bandwidth and compute, can we have a "big data + ML" machine compact enough for the home?
Was looking at it to for a kubernetes cluster and migrate existing vmware vpshere setup over, but it's really still to much home use aimed. We will settle on proxmox and VM's for kubernetes.
I have a 10KW solar install ground mounted next to my house. As long as I do most of my computing during the day it's "free" electricity. Maybe the Level1 crew needs solar on your roof.
Dropping by to say that if you haven't taken the time to upgrade to TrueNAS Scale yet, take the down-time and DO IT! All of my weird performance quirks and suddenly fixed.
I actually run one of those "NUC" type machines but it's only a quad core 4150j and 16gb ram and for the media server and phone dump sever it works just fine for my family of 4. Android phones use VLC and file manger+ and all my laptops, pcs and smart TV are able to access with no issues. 4 phones, 4 pcs, 2 TV and 1 tablet can access the same movie (1080p) at different spots with no lag. Even with scrubbing thru the timeliness of that movie. Fanless design never gets above 60c and you can't even see the cpu working more than 20-30% at our full load. That being said, it's only for Lan consumption
The modularity of ZFS is nice in that I have TNCore running in Hyper-V and TNScale in KVM on a desktop that currently dual boots Win/Linux. I don't know if this is still that case, but in an early version of Scale running an "apt upgrade" broke the whole thing. That made me wary of futzing with it outside of the WebUi, and I generally prefer just using Proxmox for VMs and containers.
I'd love to see how it works on 2 HPE Proliant P16006-001 micro servers. This seems like a great option because you can put a 10 Gb network card in both. Then you have a redundant solution with what? 12TB max of (SATA) SSD in RAID 5 with redundancy? Plus, 4 dedicated gigabit ports for VMs/containers without having to use a virtual switch. All with 8 cores for ~$1300 (plus hard drives/ssds) when they stop being $1050 each and go back to their regular $500-550 range.
FreeBSD jails do work good in TrueNAS Core, but their VM solution is really bad. I hope I can use TrueNAS Scale in future for VMs and Docker. At the moment the performance aint good.
So I've gone ALL in on unraid. I maxed out the memory in my server and i moved my GPUs into it and i run my gaming system in a VM and its all on my unraid post. Well then I learned about truenas scale... i realize its not 100% there, but im really interested... if it'd do what im doing on unraid better...
I've got an old dl380 g8 dual xeons/200gb ram with 2x25 drive Das. I currently run truenas core nested inside proxmox. I want to ditch that setup and switch to native truenas scale. This is gonna be very interesting to me. I run nextcloud and all sorts of fun things on that server and I also have a 32gb 4cores "dev" environment with consumer grade stuff (4 internal drives + 6 external drives Das) which currently runs truenas scale native for testing purposes before I make the switch...
I am kind of looking in the same direction. I bought an old 20 core xeon system for TrueNAS because my old CL2550 died (which I bought in 2014 because of Wendell :) ). I am currently running proxmox and running TrueNAS virtualized. I am kind of paralyzed by choice. Run TrueNAS vitualized? I want to add home automation stuff and nextcloud. Run that in kubernetes on TrueNAS scale? Run it in a VM in proxmox? Argh.
I'm actually running this on a Threadripper 2970WX with 128gig of ram an LSI SAS 9201-16e a NetApp DS4246 and NetApp DS2246. I have aggerated 2 10gig nic's on it and 10gig network it runs great.
For some reason I have these weird dead locks on the SMB mount where Windows explorer just freezes while the ZFS mount in Linux keeps working. It does come back after a bit. I can't find my case on Google.
Thank you for this. More on Truenas Scale and the Apps would be great. However, After being on scale from it's first release patiently waiting to get to this RC and beyond I've discovered that kubernetes is a whole different beast network wise to Docker and is quite tricky. But my biggest disappointment is sharing an Nvidia gpu on Truenas Scale (kubernetes) Apps isn't possible at the moment anyway and doesn't appear to be something on the immediate horizon from what I can find.. Gutted when I realised this and basically will have to run a VM after all, with containers or limit GPU duties to one container on Scale itself..
Thanks for that, btw. if you don't need that little box you showing a little in the video, i could put it to some good use. I run TrueNas Core as a backup target system and could see, as there's zfs in place, TrueNas scale as a snapshot target (in their means) and run multiple containers on that machine. Would be nice to see the interoperability between the 2. TrueNas core is using a configuration database to store the configuration you do on the gui, like in every aspect. TrueNas scale prob uses the same config-db mechanism. Would be nice to see tingleling with that. Like wirting config into that database to make it robust against updates or any other changes ixSystems pulls up. For now i need to reapply some cronjobs after every update, really annoying, espeacilly if you forget about that detail.
I definitely want to see how some of this goes. I've kinda watched truenas and freenas for a while, and my dad has a freenas mini that he kinda pushes to the processing limit.
Very different. Proxmox is a hypervisor to manage virtual machines and Truenas is a ZFS storage pool appliance for NAS. You can probably run Truenas scale on top of Proxmox. It sounds like some people want to attempt to run proxmox alongside Truenas. My experience with Truenas is that it’s too delicate to attempt to mix in other software. They don’t use vanilla versions of the underlying OS. They heavily doctor the OS up for their needs.
I have a question on NAS/Server setups, which is this, how easy is it to do a home server/backup solution with ready access to the storage, where you just custom build like say, an AMD APU based system with either 16 or 32GB of RAM, and could you use Windows on it, maybe a server edition, or would it be best to use a Linux distro like the TrueNAS software, and how easy would it be to set up??? Looking to eventually build a windows or Linux PC into a 1998 bubble iMac shell I have laying around, and want a great graphics card, but without the extra heat associated with it (hence an AMD APU). Also, before anyone says anything, I know the Mac thermals are trash, which is why I'm not going the dGPU route and also why I'll be trying to make a few holes in the inside of where the carrying handle goes, to increase the air output as much as possible, with a couple of fans besides... I also plan to reuse the CD bay for a laptop DVD drive, for DVD, and possibly Blu-ray, backup... Oh and I would probably end up using something more bleeding edge, like DDR5 and an APU based on RDNA 2 or 3
It's easy. Install OS, partition the disks, configure the network shares is standard stuff. I have a Freebsd, and it just works. Not in a Todd Howard way. I do DHCP server and printer server on it too. Basic stuff, everything is well documented.
@@guy_autordie ah, I'm not well versed I Linux/unix-like operating systems, although I HAVE used Linux before, and if I ever do, I tend to use LinuxMint, as that's my favorite one I've found, although I still prefer daily-driving windows for my main production and media systems, and HATE dualbooting, because of problems with the bootloader I commonly had years ago... All that said, if I get the money to build a small AMD server(specifically offline based, and "wifi without internet" compatible) I want to try an easy to use Linux/Unix/BSD system, that's more or less point and click and/or checklist style setup, where it guides you through everything needed, (if such things are possible) like it's talking to an idiot (which I'm generally not, it's just that my bad memory and slow brain hamper understanding)
I will probably go from Core to Scale. Because damn... FreeBSD is a hardcore thing. I manages some stuff have some Iocages but it just doesn't click. I like me some linux
Wendel would you be able to post more of these hobbyist nas videos?
Agreed! +1
This is probably pretty niche content, but personally I will be watching every truenas scale video you drop. Love learning more about this stuff.
Agreed. I'm happy that Wendell is making more content like this lately.
Third. I'm running truenas scale on a dual xeon server right now. I'd love to see how Wendell makes it better than what it is already.
I want to see some content on the scale side so rather than the overkill server as a NAS and virtualization stack in one box. I want to see lots of small to medium size systems “glued” together as a coherent platform
In theory it should be able to handle a failed host not just individual disks in a pool. Seeing it in practice is something I'm really looking forward to.
I run TrueNAS Core in a VM under Proxmox and just passthrough my HBA.
Seems like TrueNAS Scale could easily replace Proxmox and let me use the same storage pool directly for other VMs. Excited to give it a try when it's ready.
I've been running SCALE for 6 months now and it's rock solid stable with excellent performance. The SMB shares are really easy to work with and the Plex hardware transcoding is much easier to get going on SCALE than Core.
Yeah, I can confirm that.
I also have a couple of storage disks for SMB shares, running VMs and apps like Plex/Docker on it.
I'm using SCALE for 4 months now and I really appreciate it.
It makes so easy to share, deploy and virtualize VMs from the same GUI.
So excited to see the final release next February.
Thanks!! That answers one of my questions. I was thinking of building a box for just that. Also wanted it to run pfsense, have you tried that at all?
It would be really nice to see some testing of iWARP vs RoCE for the RDMA testing as the whole DCB overhead of RoCE is off putting for home/small-business labs.
I always watched you when you did news on that not to be mentioned channel and you peered through your two monitors to the camera before I was a teen. I've built a few NASs and heaps of PCs now and I'm super excited to see you geek out and push this new platform. That old server stuff is cheap but a few thousand dollars is a little steep (especially as a student's hobby).
My favorite practical home lab thing has been running my nas and virtualizing all my home services with docker. More videos like this. 👍
I've been running SCALE since the alpha and I have no major complaints. I'm waiting for the stable release before I migrate my synology's data over but for now I've got pihole, plex, jellyfin, multiple valheim (game) servers, minecraft (game server), and transmission (torrents) running via containers.
I wanted the highest core count lowest power usage system so I've got:
Ryzen 9 5900 12 Core @ 45W TDP via "Eco mode"
Asrock Rack x570 mATX motherboard - Future 10GBe expansion
64GB 3200Mhz ECC (i really want another kit before I add spinning rust storage)
NVME only pool for now.
You run my dream homeserver setup =D. I like especially the Asrock AM4 Serverplattform, with the KVM built-in. Do you have the 10 Gbe network version of that board? Im surprised, that a Ryzen 3000 Series or 5000 Series could run with such low powerconsumption. Or does it? I assume it from your Eco Mode.
@@MrUltranobody Yes I have the version with 2X 10GBe, I just don't have anything to use those NICs with yet.
And yes, the Ryzen chips support ECO mode.
65W TDP Chips (like the 5900) -> 45W
105W TDP Chips (5900X/5950X) -> 65W
Yeah, I know that feature of Eco Mode from the Ryzen 5000.
But how much is your power consumption in Idle?
By the way, you can get 10 GiB Switches for around 100 Bucks.
Love it! I am a COMPLETE NOOB and having freenas running for the last 5 years.... Brings such happyness having ZERO issues with it... Well, it BURNS USB memory sticks if u try to zfs them... but its all good. Hahaha
Keep up the good vid's.
Wait, you were adding USB sticks to the ZFS pool? That's absolutely wild
Would love to see a video with you and Tom @lawrence do a video or at least see you guys on a podcast together.
that was one of the friendliest introductions to a new project i've ever seen. especially the part in keeping the support work away from the original developers of truenas (core), i hope that is as appreciated by them as it is by me. i've started tinkering with scale in a vm (on my core machine) just to figure out how everything works. still struggling with how to set up smb shares in a "advanced home network" environment, so no AD or similar to get the windows security identifier... very strange compared to core. but i'll get used to it and i think it'll be amazing as a "one fits all - hyperconvered" infrastructure thing where i can run everything i need on one box. looking forward to that series of you!
I just build a TrueNas core using a HP Workstation, will give the TruneNas Scale a try, still learning TrueNas, thanks for sharing
This is excellent! I would LOVE to see a video on setting up VPN/next cloud. Love all this info, looking forward to this video series!
You magnificent so and so... I have everything but a CPU cooler for my upcoming truenas scale build and i wanted to do some advanced VM stuff as well (preferably clustering. I was hoping to get proxmox installed along side truenas scale given they both seem to be built on debian iirc). I want to be able to easily pause, move a VM to a different box, and resume a VM while using all the great filesystem management, docker, and smb sharing awesomeness in truenas scale.
Been contemplating on building a truenas machine to replace my unraid. TrueNAS has been looking real good and native zfs is something I want.
I've been running freenas for a long time. I just upgraded to truenas last night. 18 core xeon 128GB of ram and i'm only using it for SMB. I've been having reliability issues with permissions since I started messing with it... I wish it was a bit easier to manage.
Just build a Home NAS with TrueNAS Scale two days ago, very keen to see where this series goes.
Could you do a noobs guide on TrueNAS? You're really calm and easy to listen to, but I feel like Im so out of the loop with some stuff that I would love to learn more from you
I'm really excited to test out the actual scaling part of true nas scale.
can't wait to see the upcoming videos
I'm running TrueNAS Scale on my old gaming PC, R7 2700x, 32GB of RAM and some IronWolf 4TB HDDs. Works pretty well. This is why you don't throw away old hardware if you can
Wendell, I have to admit that I am looking forward to this series of videos.
Can’t wait for the rest of this series. Currently acquiring the parts to build a 42u dedicated to AI research with 100gb infiniband between everything
You mentioned Windows Server 2022 performance, is that something you're planning to do a video about?
Really good video - thanks a bunch. TrueNAS SCALE is indeed a cool product, though I think most users will be happy with TrueNAS Core. The main attraction to SCALE are the Docker containers and KVM integration, though the product was initially developed to solve horizontal scaling - a feature that TrueNAS or FreeNAS could never quite deal with in the past. Being able to have native Docker containers I think will make the platform seem more appealing to new users, but for the most stable experience, TrueNAS Core is likely still the tried and true, and should still be the base of the TrueNAS Enterprise products (not necessarily the same build, Core is kinda the beta for Enterprise, but mostly based on the FreeBSD platform with more native ZFS support).
Interesting to try: run an Elasticsearch cluster on it with machine learning and something with the complete project Gutenberg ebook collection on it.
I've been using Scale since early beta, and it's been pretty solid. Using NFS/SMB/iSCSI without any issues. Had some (possibly) self-inflicted routing issues that I've worked around. Not a fan of their wizard/GUI for creating a container, hopefully they add support to dump docker-compose files similar to portainer.
Since Portainer is a Docker container, can’t you run it in TrueNAS Scale?
The appliance nature of TrueNAS makes it really easy for my client to manage users.
I set it up for them in a VM on a DRBD partition, 2 server solution.
Ridiculously cheap, and redundancy beyond what some of the large enterprises i work for can manage.
I am slightly worried about the recovery capability of ZFS vs EXT4 in the event of a drive failure & DRBD failover.
Along the lines of your drives saturating the memory bus, can you use something like Intel MBM to report memory bandwidth usage? Note that MBM didn't work on certain versions of Intel CPUs (I know right!?).
I use a DAS as I need fast local high capacity storage. My Switch doesn't have anything faster than Gigabit speeds, and anything faster was prohibitively expensive when I made the purchase. I ended up with a QNAP TR-004 because it has a 5Gb USB 3 connection. Faster than my network, but future proof so when I upgrade and get a faster switch and a NAS I can connect it as local storage expansion. (the TR-004 also gives me hardware Raid 5!)
I've rarely seen anyone talk about this advantage.
Awesome! I use a VM running k3s/Rancher to manage all my containers on TrueNAS Scale. I don't like what they've done to the UI for apps and I only run it when I need it. Before, I ran Rancher right on TrueNAS Scale but that caused some problems that grew out of controls after updates
Wendell: The Anti-Grinch
What timing! I'm currently migrating my dual E5 v2 128 GB build to TrueNAS Scale. 4x 2 TB NVMe in RAID Z1. Backups to B2. Everything's great so far. Hourly file history for windows desktops small oopsies, two week ZFS snapshot window for normal oopsies, daily offsite one-ways for big oopsies. zstd-3 and dedupe can run at my line rate (10G).
I have yet to play with iSCSI for steam library, but it's on my list. It's so nice to finally have a high performance and reliable local VM host for my software playground wants.
Thanks for the vid! I have a pair of R820's with SFP+ 10G cards that I'll be using to test scale on...mostly targeting k8s and docker functionality
There's a lot of low frequency noise in the sound, I think from bumping into the table or something. Maybe have a look at isolating the mic from the table better? Now I keep thinking someone's knocking on my door :P
running it on mini x+ could not be happier used only for family nextcloud and so far the upgrade process have cause minor issues other services i have running heimdall , home assistant and traefik to which i can't understand at all yet since its more capable than nginx but lacks a gui . just a note in case other users of scale are reading
if you will install traefik or similar remember to change your trunas scale ports, at least for me when i try to edit config.php for nextcloud need to do a apt-get update then install every time nano so i can adjust the file.
cheers looking forward for next video
I'm super excited about TrueNAS Scale and this video series. Would love to see how to configure ZFS for good performance with spinning rust and an SSD or two.
I just put a DS4246 into my LACK rack, need to fill it but the network is 10g and ready for me to get this all setup. Just need the time now..
Eyeohs lol - always makes me laugh when people make words out of Acronyms - hadn't heard that now before!
For RDMA look at what Oracle is doing on their Engineered systems/platforms.
Will there be any "scale out" content in the future; ie multinode TrueNAS Scale node testing? I can't recall if i've ever seen someone using the scale out feature (Gluster?) in Scale, so that would be interesting, especially failure modes & recovery
Can we stand up a DIY video streaming service with that type of bandwidth? (Kind of like "guerilla RUclips") Or is that more bottle-necked at the network bandwidth?
Or, with that much data bandwidth and compute, can we have a "big data + ML" machine compact enough for the home?
Tailscale can get around the carrier grade nat problem
Was looking at it to for a kubernetes cluster and migrate existing vmware vpshere setup over, but it's really still to much home use aimed. We will settle on proxmox and VM's for kubernetes.
I like this type of content. More please !
I like this, I'll check the forums next time I'm bumbling on some vague perfectionist home IT improvements
I have a 10KW solar install ground mounted next to my house. As long as I do most of my computing during the day it's "free" electricity. Maybe the Level1 crew needs solar on your roof.
Subwoofer users, don't worry, he's just hitting microphone :D
Dropping by to say that if you haven't taken the time to upgrade to TrueNAS Scale yet, take the down-time and DO IT! All of my weird performance quirks and suddenly fixed.
ZFS will be better with ECC, make sure your small form factor support ECC if going SFF (Ryzen Pro)
Yes, this is very interesting! Take us down the rabbit hole :)
I would love to see your breakdown/analysis of how VMs run on TrueNas Scale. Really enjoy your videos!
I actually run one of those "NUC" type machines but it's only a quad core 4150j and 16gb ram and for the media server and phone dump sever it works just fine for my family of 4. Android phones use VLC and file manger+ and all my laptops, pcs and smart TV are able to access with no issues. 4 phones, 4 pcs, 2 TV and 1 tablet can access the same movie (1080p) at different spots with no lag. Even with scrubbing thru the timeliness of that movie. Fanless design never gets above 60c and you can't even see the cpu working more than 20-30% at our full load. That being said, it's only for Lan consumption
The modularity of ZFS is nice in that I have TNCore running in Hyper-V and TNScale in KVM on a desktop that currently dual boots Win/Linux. I don't know if this is still that case, but in an early version of Scale running an "apt upgrade" broke the whole thing. That made me wary of futzing with it outside of the WebUi, and I generally prefer just using Proxmox for VMs and containers.
I didn't know that RDNA had this much potential!!!!
I'd love to see how it works on 2 HPE Proliant P16006-001 micro servers.
This seems like a great option because you can put a 10 Gb network card in both.
Then you have a redundant solution with what? 12TB max of (SATA) SSD in RAID 5 with redundancy?
Plus, 4 dedicated gigabit ports for VMs/containers without having to use a virtual switch.
All with 8 cores for ~$1300 (plus hard drives/ssds) when they stop being $1050 each and go back to their regular $500-550 range.
I would be interested in seeing how to cluster multiple true nas scale nodes. Pcie passthrough would be interesting for vms as well.
FreeBSD jails do work good in TrueNAS Core, but their VM solution is really bad. I hope I can use TrueNAS Scale in future for VMs and Docker. At the moment the performance aint good.
You know Wendell is becoming serious when he finally add another server to his rack.
So I've gone ALL in on unraid. I maxed out the memory in my server and i moved my GPUs into it and i run my gaming system in a VM and its all on my unraid post. Well then I learned about truenas scale... i realize its not 100% there, but im really interested... if it'd do what im doing on unraid better...
15:00 Just a thought, could it be achieved with on the same server, ZFS -> ISCSI -> mount a drive in Windows Server VM -> SMB RDMA?
On the right track. No iscsi pass through a virtual fs object.
I've got an old dl380 g8 dual xeons/200gb ram with 2x25 drive Das. I currently run truenas core nested inside proxmox. I want to ditch that setup and switch to native truenas scale. This is gonna be very interesting to me.
I run nextcloud and all sorts of fun things on that server and I also have a 32gb 4cores "dev" environment with consumer grade stuff (4 internal drives + 6 external drives Das) which currently runs truenas scale native for testing purposes before I make the switch...
Looking forward to some “pushed to the limit” TrueNas setups
My home ISP upgraded my upload from 20 to 120 megabits... so needless to say, any wifi i every join other than home is now on demand to use my VPN!
I am kind of looking in the same direction. I bought an old 20 core xeon system for TrueNAS because my old CL2550 died (which I bought in 2014 because of Wendell :) ). I am currently running proxmox and running TrueNAS virtualized. I am kind of paralyzed by choice. Run TrueNAS vitualized? I want to add home automation stuff and nextcloud. Run that in kubernetes on TrueNAS scale? Run it in a VM in proxmox? Argh.
I would watch more TrueNAS content.
“But what about unraid?”
unraid isn't worth even mentioning in the same sentence as zfs
I'm actually running this on a Threadripper 2970WX with 128gig of ram an LSI SAS 9201-16e a NetApp DS4246 and NetApp DS2246. I have aggerated 2 10gig nic's on it and 10gig network it runs great.
For some reason I have these weird dead locks on the SMB mount where Windows explorer just freezes while the ZFS mount in Linux keeps working. It does come back after a bit.
I can't find my case on Google.
I'm using several Gigabyte NUCs (BRICs) for servers. They work great.
Don't go chasing waterfalls, stick to the Docker containers you're used too...
Am I the only one that heard that finger drumming and went to check who was knocking at their door?
Thank you for this. More on Truenas Scale and the Apps would be great. However, After being on scale from it's first release patiently waiting to get to this RC and beyond I've discovered that kubernetes is a whole different beast network wise to Docker and is quite tricky.
But my biggest disappointment is sharing an Nvidia gpu on Truenas Scale (kubernetes) Apps isn't possible at the moment anyway and doesn't appear to be something on the immediate horizon from what I can find.. Gutted when I realised this and basically will have to run a VM after all, with containers or limit GPU duties to one container on Scale itself..
Wendel, it is fascinating to hear you ramble, though my comprehension of what you are putting out I would be ashamed to disclose.
Thanks for that, btw. if you don't need that little box you showing a little in the video, i could put it to some good use.
I run TrueNas Core as a backup target system and could see, as there's zfs in place, TrueNas scale as a snapshot target (in their means) and run multiple containers on that machine. Would be nice to see the interoperability between the 2.
TrueNas core is using a configuration database to store the configuration you do on the gui, like in every aspect. TrueNas scale prob uses the same config-db mechanism.
Would be nice to see tingleling with that. Like wirting config into that database to make it robust against updates or any other changes ixSystems pulls up.
For now i need to reapply some cronjobs after every update, really annoying, espeacilly if you forget about that detail.
This is great!! I'm curious how setting up some kind of HCI at home would be
I've been wondering whether adding LXD to TrueNAS Scale would be a possibility.
"yes" but officially no but of you do what I did sure. Tho I'd suggest migrating to podmans machinery for sanity
cannot wait for more videos on this...
DDR5 on truenas would be interesting, since you talked about memory bandwidth
Would love to see a series on a Home NextCloud Server running on a ProxMox and VPN'ing in.
I have a NUC 10 NUCi7FNH. So you think I should go TrueNAS Scale or Core? Maybe I will do this and and run Plex container
I definitely want to see how some of this goes. I've kinda watched truenas and freenas for a while, and my dad has a freenas mini that he kinda pushes to the processing limit.
I hate cockpit! I think I prefer have truenas as NAS only... and then for hypervisor, PROXMOX
i love cockpit - been porting over to podman now. 🙂
Can't wait for Wendell to tackle Openshift
What's the differences between TrueNAS Scale and Proxmox, and what should be the use case to use one over the other? 🤔
Very different. Proxmox is a hypervisor to manage virtual machines and Truenas is a ZFS storage pool appliance for NAS. You can probably run Truenas scale on top of Proxmox. It sounds like some people want to attempt to run proxmox alongside Truenas. My experience with Truenas is that it’s too delicate to attempt to mix in other software. They don’t use vanilla versions of the underlying OS. They heavily doctor the OS up for their needs.
Is it just me or is there a teeny-tiny delay in the audio vs the video?
... maybe
Like you say, a teeny-tiny delay.
Most of there vids lately have had it, dont know why.
Somebody give Wendel a cookie and a glass of milk.
Computer janitor is uderrated profession until you hit a wall.
Can you dive in to k3s on truenas scale and how it works in a cluster?
I have a question on NAS/Server setups, which is this, how easy is it to do a home server/backup solution with ready access to the storage, where you just custom build like say, an AMD APU based system with either 16 or 32GB of RAM, and could you use Windows on it, maybe a server edition, or would it be best to use a Linux distro like the TrueNAS software, and how easy would it be to set up??? Looking to eventually build a windows or Linux PC into a 1998 bubble iMac shell I have laying around, and want a great graphics card, but without the extra heat associated with it (hence an AMD APU). Also, before anyone says anything, I know the Mac thermals are trash, which is why I'm not going the dGPU route and also why I'll be trying to make a few holes in the inside of where the carrying handle goes, to increase the air output as much as possible, with a couple of fans besides... I also plan to reuse the CD bay for a laptop DVD drive, for DVD, and possibly Blu-ray, backup...
Oh and I would probably end up using something more bleeding edge, like DDR5 and an APU based on RDNA 2 or 3
It's easy.
Install OS, partition the disks, configure the network shares is standard stuff.
I have a Freebsd, and it just works. Not in a Todd Howard way. I do DHCP server and printer server on it too. Basic stuff, everything is well documented.
@@guy_autordie ah, I'm not well versed I Linux/unix-like operating systems, although I HAVE used Linux before, and if I ever do, I tend to use LinuxMint, as that's my favorite one I've found, although I still prefer daily-driving windows for my main production and media systems, and HATE dualbooting, because of problems with the bootloader I commonly had years ago...
All that said, if I get the money to build a small AMD server(specifically offline based, and "wifi without internet" compatible) I want to try an easy to use Linux/Unix/BSD system, that's more or less point and click and/or checklist style setup, where it guides you through everything needed, (if such things are possible) like it's talking to an idiot (which I'm generally not, it's just that my bad memory and slow brain hamper understanding)
Storage spaces are so bad :( How do they just churn out storage shit for so many years and not get called out on it.
Syncthing on truenas seems to offload almost all of the computation to the clients
I will probably go from Core to Scale. Because damn... FreeBSD is a hardcore thing. I manages some stuff have some Iocages but it just doesn't click.
I like me some linux
I'm waiting for this videos ...
Yes, 100% video on TrueNas Scale VPN
And I'm still clunking around a dual xeon e3 with a LSI mega raid 72TB raid 60 at the house
We are waiting on your Nextcloud with open VPN video
Curious how TrueNAS compares to Proxmox.
Wendell please do the vpn into nextcloud video. Well timed with my eBay thinkcentre m93p mini purchase