Can TrueNAS Scale Replace your Hypervisor?
HTML-код
- Опубликовано: 20 окт 2024
- Thanks to Linode for sponsoring this video. Visit linode.com/Cra... to get a $100 60-Day credit when signing up for a new account.
TrueNAS has always been a great file server, but wasn't built with virtualization support in mind. With TrueNAS Scale, that's about to change.
But first... What am I drinking???
From WellBeing Brewing comes a non-alcoholic Golden Wheat Ale, the Heavenly Body. With only 68 calories, it's definitely a great alternative to grabbing a soda, but I don't think it'll replace beer in my fridge anytime soon.
Links to items below may be affiliate links for which I may be compensated
Check out parts from my TrueNAS Scale Server:
Supermicro 846 Chassis: ebay.to/3pAilW1
Supermicro H11SSL-i SP3 Motherboard: amzn.to/3xtSkKG ebay.us/N73DGi
32GB DDR4-2666 ECC RDIMM (x8) - amzn.to/3zDledg ebay.us/rZwRv8
8TB HGST He8 Helium SAS: amzn.to/2X7Xb7C
Seagate FureCUDA 1TB NVMe: amzn.to/3iOOIij
Seagate IronWolf 110 1.92TB SATA SSD: amzn.to/3cNdKKG
EVGA Supernova T2 1600W: amzn.to/2U9cjjb
Noctua NH-U12S TR4-SP3: amzn.to/3q1lZby
Fractal Design Define 7: amzn.to/3iQJeUe
AMD Epyc 7601: amzn.to/3iKYNg5 ebay.us/D9ioE9
Find the parts I recommend on my Amazon store: www.amazon.com...
Follow me on Twitter @CraftComputing
Support me on Patreon or Floatplane and get access to my exclusive Discord server. Chat with myself and the other hosts on Talking Heads all week long.
/ craftcomputing
www.floatplane...
Music:
Lobby Time by Kevin MacLeod
Link: incompetech.fi...
License: filmmusic.io/s...
Would love to see you compare Truenas Scale vs. Proxmox
Some benchmarking and comparisons would be great. I feel like TrueNAS is nearly there for people that have easier setups
what is there to benchmark, it's Debian Linux+KVM+ZFS, it will run more or less the same as Proxmox, which also does the same
@@marcogenovesi8570 Breaking news: Truenas Scale's zfs performance is slower than Core's.
Benchmarks are useful...
Just because it should be the same in performance, doesn’t mean it is.
@@Clarence-Homelab Breaking news: Truenas core is FreeBSD and ZFS on BSD has always been faster than ZFS on Linux (what Truenas Scale is). Benchmarking well-known stuff is waste of time.
Truenas scale is just Debian with Truenas web interface
Tested personally, on same machine: Proxmox Win10 VM vs TrueNas Scale Win10 VM. Reults: same CPU score, memory score, etc..., no diff. In background were running additional 5VMs just to have "some load"
True (mostly (extremely (very)) boring) story: The only reason why I have a homelab is because I decided to build a FreeNAS 10 host because I thought corral looked rad and because I could run most of my workloads in provided Docker support. In the time between me ordering my hardware and receiving my hardware, it was announced that corral would no longer be a thing. Then I realized I ordered LGA2011 v4 CPUs (which I didn't _want_ to return) and a v2 mobo (which I _couldn't_ return) so I did the only logical thing and ordered a v4 mobo and v2 CPUs ... the LGA2011v2 rig ended up being my FreeNAS 11 rig and the v4 a XCP-ng rig, but with TrueNAS Scale, it's looking like I can go back to that original single host dream. Heck! With that GPU passthrough, I can even have a Windows VM I can RDP into so I can continue to pretend to know how to use Fusion 360!
What CPUs are they and how much ram and storage HDD and or SSD do you have?
@@time-alinge
TrueNAS rig:
dual Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz
64G of memory
10x8G HDD
4x128G SSD (the underlying freeBSD chokes while booting with these installed, so I just use these when I need some temporary, fast storage).
XCP-ng rig:
dual Intel Xeon E5-4620 @ 2.10 GHz (I got these as used and `/proc/cpuinfo` reports the CPU model as "0000", so probably a kock-off, modded, or engineering sample)
256Gb of memory (turns out, this is overkill for my workloads)
500GB local storage (10GiB link back to TrueNAS)
I love hearing long nerd homelab stories like this
It took a RUclips video and a garage sale for me to decide I wanted a Plex server and it's all history
I truely did not understand what you were exactly trying to say until I red this comment three times. Partially because you entered so much text between "true story", so I thought it was a reaction to something.
Anyways you should not create such long sentences, nor use "because" or equivalent twice in a sentence. Sometimes it is better to just use a dot instead of a comma and use a couple extra words.
I hope this makes sense and have fun working on your homelab ;)
@@lucappelman8558 Because I wanted to tell a true story, I entered in a lot of super (unhelpful (irrelevant (but actually factual))) information about the events and consequences of failing to do a detailed amount of compatibility analysis of Intel socket design, layout, and notch positioning.
The AHCI Emulation does not limit speeds, it just emulates the storage calls. So, you can go over 6gbps.
Similarly, it is theoretically possible to go over 1Gbps on the Intel e1000 virtual NIC. If you transfer data between VMs, you will probably see it break 1Gbps since the data only goes through server memory.
That said, in most cases VirtIO is faster and has a lower performance penalty.
I installed Scale a week ago for the first time, thanks for the heads up on the VM section, answered a few questions I had. Good work.
My favorite channel since I found it months ago! Thanks Jeff
One correction: Emulated AHCI is not limited to 6Gbps. Early PCIe SSDs were AHCI instead of NVMe. It just introduces the overhead of emulated AHCI storage calls instead of block level access.
He makes similar claim about emulated NICs which I also assume is false. Of course, they have higher overhead, but they are not limited to 1Gbps.
@@nmihaylove correct, you can have emulated 100Mbit NICs that actually go at 10Gbit or whatever is the actual max interconnect speed of the virtual network (VMWare or KVM)
Very much looking forward to more of your videos on TrueNAS scale! hope there's one covering containerization features next.
Do really want to see a VirtIO video. Especially with windows server 2022. (I think the NIC driver was not compatable last time I tested it.)
For way TrueNAS VM not use dynamic memory, I think it's mostly due to ZFS Cache. Unlike most hypervisor which always have free memory, ZFS will comsume all the memory for caching then VM will need to keep swap shared memory with ZFS that could cause performance issue.
delightful color setting, my eyes thanks you from my dark bunker
I would like to see a video on clustering, both of data and of VMs.
I've never understood the fascination with dynamic memory allocation. I've never had good luck with it, and I subscribe to the old adage "allocate what you need to the VM, no more...no less"
Thanks! And yes to review all the VM options.
Btw I love the beer glass merch, I ordered one last fall.
Yes doing some more truenas scale VM videos. Comparisons, performance benchmarks, etc!
Great timing. As Scale comes out of beta I am really intrigued by the idea of combining my Unraid and TrueNAS core server into a single TrueNAS scale server.
Unraid is great but I prefer managing Docker containers via Docker-compose. I use VMs a bit, but mostly to try out different OSes, not to provide a service such as plex
I really fucking like your content. The vibe is immaculate and you see me like an IT pro's Techie
Excellent explanation. This was really impressively done. Much higher polish than a year before.
Great video! Hoping to see a release of the Performance Benchmarks done and comparisons once they are available!
It has a few interesting possibilities. With KVM, docker (also with gpu passthru), and using ZFS features it also has nice backup and replication built in, all free.
Great video! I've been learning about TrueNAS and struggling through the learning process.
I love this kind of video going through some of the finer points of the system!
Jeff's beer is a better progress bar than what Windows has.
What is the trackball mouse you use, kind sir? I looked in your amazon parts list, but I couldn't find it, likely because I'm cursed with not being able to see things right in front of me, but I would love to try it out.
Great video Jeff, TrueNAS works very well for NAS storage!! Thanks for the video!!
Awesome video! Have you got/will there be an upcoming video on TrueNAS Scale apps?
Worth it for the Archer reference.
Thanks a ton for the great content. I have found your videos quite helpful as I find my way around this “new world” of self-hosting / home lab setup.
In a Proxmox + TrueNAS or OMV setup, what is best approach for ZFS Storage Pool. Is it best to setup the zpool in Proxmox for use by the NAS software or is it better to setup the zpool from within the NAS software?
Tried to set up a Windows VM and I was wondering why the hell it was so (unusably) slow. Turns out it doesn't like AHCI and runs way faster on VirtIO
Been finding these TrueNAS Scale videos interesting, especially as after updating to TrueNAS-12.0-U8 this morning I now see that there's a way to update to TrueNAS Scale. I think I'll be holding off for a while but nice to see they are going to provide a way to upgrade.
I’m in the same boat. Been on FreeNAS/TrueNAS Core for probably 8 years. Learned more than I’d ever thought I’d learn about BSD (as a rank amateur of course). Switching to Linux based sounds awesome for compatibility, as I’ve spent loads of time getting some simple things running in BSD jails. Migrating or redoing my jails to VMs in Scale sounds like a pain!
The joke is on you. You've always had to bribe him. He just keeps altering tthe deal, pray he doesn't alter it any further.
Great video, thank you for all the content, I have my truenas file server and pihole all thanks to you.
Debian is a contraction of Deb (Ian Murdock's college girlfriend), and Ian (Ian Murdock). Deb-ian, not Dee-Bee-in.
Maybe some Docker content on truenas scale =) Would be nice =D
Yes, I would like to see more about the thing you said we should comment about to see more of.
Is there any downside to running TrueNAS as a VM within Proxmox?
+1 for virtIO drivers performance video😀
If/When Unraid natively supports ZFS, that will be my ideal hybrid NAS/Hypervisor/Docker host.
OMV with proxmox kernel and zfs works very well. If your able to run a proxmox kernel in unraid then thats your solution.
still using 2 Unraid boxes. works flawless
Dynamic memory size is available by now^^ you will have a "minumum memory size " field where you can put any value smaller then the max size and it will automatically adjust ^^
Good TrueNAS video great beer review
Thanks Jeff, great stuff as usual. Could you do a Video on running OPNSense in a VM on TrueNAS Scale please?
I'll stick with Proxmox for most of my VMs, but I do have a few simple services running in TrueNAS VMs. Mostly things that interact with the NAS directly. Mainly a torrent seedbox for my uh... "Linux ISOs".
It is nice to save resources on my Proxmox host when the TrueNAS box has more than enough spare resources to pull double duty.
I know this is an old video, but what are thoughts on running Home Assistant OS as a VM in TrueNAS scale? I have 32GB RAM and 4x4TB in RAIDZ1 with 500gb Samsung Evo SSD setup as a cache drive (sometimes I will run virtual machines directly over the network, so the 32GB ram seems to like having the cache there when I load up a VM). CPU is an i5 4400, so more RAM would be an entire platform upgrade and aside from the storage drives, this was just a fun project I cobbled together with old hardware I had laying around...
Talk about timing ... I just had some experience with this over the last week. Worked well depending on what I needed the VM to do. I did however try creating a glusterfs cluster using some vms across a few Truenas Scale systems and well ... the performance wasn't great.
To be fair performance maybe would have been better with better hardware.
I'll let u know that wanto see a video comparing the two disk drivers. Oh, I might add that you should try out the Balast Point Manta Ray DIPA. Hard to come by at my side of the pond, but when you do... oh, there is a sweet moment of joy. Or the St. Austell Big Job from the UK, might be hard to get hold of in the US, but its a treat.
Jeff, you're mixing up virtio and SRV-IO. Virtio devices and drivers are just simplified and more efficient ways for the guest and host OS" to communicate rather than emulating real hardware. The guest needs to have drivers to use it but there isn't anything special needed on the host.
SRV-IO is virtualization built into the underlying hardware device that exposes multiple logical endpoints to the host that it can then pass through to guests. The guest then talk directly to the hardware's special guest logical endpoints, but it does require some cooperation from the host OS' drivers to function. It also completely prevents migrating a VM to a different host or any fancy networking or storage shenanigans that the host OS' might support. So no software bridges, VPN tunnels, LVM or ZFS snapshots, etc. It should be faster on the high end, but boy what a price to pay.
its Single Root I/O Virtualization (SR-IOV), not SRV-IO, and there are ver few HW outside NICs supporting it. VFs (virtual functions) are then passed through the host OS to the guest, just like with normal PCI devices. IT also means same limitations apply as to any other PCI passthrough VM would.... where it really make sense is high performance networking, usually acompanied by DPDK based software implementation for routers, LBs etc..
"it's only 68 calories, so i guess it has that going for it." when you have to reach THAT far, just pour it out haha
Dropping by to say that if you haven't taken the time to upgrade to TrueNAS Scale yet, take the down-time and DO IT! All of my weird performance quirks and suddenly fixed.
What about the other way around? Maybe you want something a bit more robust than open media vault but could just run Truenas scale as a proxmox container or VM as you'd get the more robust features but for a home lab, still manageable inside that.
Imo this is only worth doing if you can pass through your storage controllers
You should do a video on the clustering with true command
I believe it uses gluster
I have a Win10 VM running on TrueNAS Scale. I'm passing through a Quadro M2000 and using the Parsec setup you recommend in one of your gaming server videos. It runs benchmarks like the unigine heaven in full screen at 1080p. I can play Cities Skylines and some other games. ✌🏻
Tried to search your channel but could not find anything about ECC memory regarding to ZFS, maybe an topic for some future video?
Jeff, I love you to do video about the docker part ( apps ).
I'd really love to see you try to setup an opnsense VM in TrueNAS Scale!
I'd recommend the Einstök Icelandic White Ale with that ;)
I'd love to see a review of how TrueNAS Scale deals with Docker as that is a particular shortcoming of Proxmox.
I would really like to know what you think about truenas scale in its current state. Can it do what needs to be done for VMs and containers. Should we be switching to bluefin? Or should we be using proxmox and truenas (core or scale not sure which) in a VM?
I've been using TrueNAS Scale as my hypervisor for a few weeks now. I was able to get a Windows VM with GPU passthrough running after some trial and error. Two things about TrueNAS Scale that are still bothering me: It doesn't automatically create a network bridge so that VMs can access other services of the hypervisor, and with the built in OpenVPN Server there doesn't seem to be a way to create a VPN tunnel where my iOS device can connect with VPN and see other hosts on my network, it can only see services of the TrueNAS server. I know one could use static routes on their network gateway, but mine doesn't have that feature and I'm not ready to rebuild my network.
I run my Servarr tools in a VM running on my truenas core machine and it is super stable. Excellent if you need one or two self hosted tools. Alot more flexible than the plugins/jails avaliable
What would you recommend, i only have one computer and want to run a NAS and Server (win svr 2016) on the same machine, would you recommend running 2016 with hyper-v and a truenas vm or a trunenas installation with a windows server vm?
VirtIO seems like a good topic for a video 👍
Can you please make a video from the perspective of a server noob. I have clambered myself together a Freenas server - my next plan (debate) is whether to make a new server with a hypervisor with VMs (freenas and the new services I want) or keep freenas as just server and a couple of VMs running on it that run things like pi hole.
How did you get the P4 GPU to show up in the TrueNas environment?
I think the whole "fixed" element of VMs on TrueNAS goes back to it originally being built off of VirtualBox. Most of the issues with it could've been fixed by migrating to just about any other virtualisation solution, but as you said, it was never meant to be a platform for VMs. The FreeBSD → Linux "move" is just a flavour though, to appeal to management types who, in the past, would've been than the "nobody ever got fired for buying IBM" camp.
The FreeBSD to Linux move has been done because that way they get KVM hypervisor, docker containers, a bunch of cluster filesystems and a lot better hardware compatiblity.
TrueNAS Scale has done more progress towards their "vision" for Truenas in a year than Truenas (on BSD) has done in 10 years, mostly because all those things are already turnkey on linux ecosystem
I've got a home assistant VM running on TrueNAS (not scale though). Had it running on a Pi but it kept eating SD cards, so I VM'ed it.
Been contemplating the jump from Core to Scale, myself. Wondering if there are any performance differences still, and also would love to see Scale with a Plex VM w/ video card passthru for hardware encoding. Might get to retire/repurpose my Hyve Zeus! :)
I'm using Scale Plex "app" with GPU passthrough and it works fantastic. Since it's kubernetes you can have it migrate easily to other hardware in the future if you need to as well. I'm keeping an eye out for Nvidia's next (ampere based) SBC for that purpose.
@@snives7166 Oh nice; they're using kubernetes for the plugins/"jails"? I presume that means it'll stay updated more frequently than the previous versions
@@shoebucket the plugins are kubernetes containers, and there is already a large community making them, it's the project "TrueCharts".
Thank you very much for this interesting video. How would you rate the maturity of the OpenZFS implementation on top of Debian, compared to ZFS on FreeBSD? Do you totally trust the data integrity of the "early versions" of TrueNAS SCALE?
i am running zfs on debian for years using proxmox kernel, which is in my opinion the best option, don't know if thats possible with truenas
9:48 is price less lol!! nice video! thanks!
I have a question, Do I need 2 discrite gpu for my Truenas in creating virtual machine?
I'd definitely like to see performance difference of both disk and network drivers.
I'd also be interested in your thoughts about Scale VMs vs. something like XCP-ng.
I would like to see a fair comparison between running proxmox or TrueNas Scale, with benchmark. Is there even a challenge?
Second, but again at 54 seconds ago? I swear I keep launching RUclips right as you upload.
And I use TrueNAS SCALE too!
Yeah, I have a script that publishes a video everytime you login. Pretty annoying, but I've got to feed the algorithm somehow.
What do you attribute the file performance difference to between Scale and Core and do you anticipate a closing of that gap?
Great run-through, thanks.
Just to ask, is anyone else hearing high-pitched background noise in the video?
yup only on headphones
ye. I'm hear
Given the stable version is to be released 22.02.2022 ;) you might have waited a few more days to get more stable/polished version for a more production-ready environment...
I really REALLY want to like truenas, the new docker/k8s "app" system is really nice... Bit for some reason it is slower then, for example, running docker in a vm or lxc container... I will try it again in a year i guess, as i always do :-D
“If you want to run VM’s on truenas in a production environment”
*chuckles* I’m in danger
With the fact I'm looking at doing a full rebuild/update of my home server I would love to know more. Cause might completely change my mind on the set-up .
I want to know when trueNAS Scale reaches feature parity with UnRaid, then see some folks comparisons and opinions on them.
With Truenas scale coming along quickly and Unraid getting ZFS support soon, it will be a very interesting comparison.
Very interested in the deep dive on vma om truenas. I am currently looking what would be better, running vms on truenas or run truenas as a VM on VMware.
What I also would like to see is your security posture. I can only imagine your network being under attack daily just for being a public figure.
Looking forward to more content. As always I am your viewer, tinus 😉
Scale is all about those k8s apps... so much simpler care and feeding than VMs, especially once you add the TrueCharts catalog
On How about a video on how to enable PI HOLE in TRUENAS SCALE Thanks ! Thanks for your video. They make life easier.
IX should focus IMO on the continued development of Core, and FreeBSD and Byhve. I get the distinct impression they're going to see if Scale becomes more popular and drop Core, or vice versa. I always love how Core is not Linux based.
Would love to see something about trunas clustering and vm migration between hosts.
I was considering the exact setup you mentioned with proxmox managing all storage but it just sounded like a pain. If truenas can do this I think it’d work better for me.
proxmox works great and easy to setup and you can actually run OMV in VM which already for years runs on debian, but now TrueNas Scale is debian its worth to keep an eye on it, but i still love proxmox so will be hard to switch, TrueNas would need to step up the game first for the VM settings.
can you mount an existing KVM machine to scale, or migrate from a different scale server?
Video about docker integration would be really nice... I am currently running Truenas core on my NAS, but lack of docker there really hinders me
I would be very interested in seeing the difference between the VM's drive subsystem configuration
would be very interesting in showing how to virtualize a macOs machine in Truenas Scale
Added support of PCI passthrough has me intrigued. My only dilemma is I want to be able to learn VMware since it's the industry standard. Also the servers I have don't have the right raid controllers to make truenas happy.
Go hyperconverged. Run esxi or proxmox and pass through your hba hardware to a vm and run truenas there. Then use truenas as a nfs for the hypervisor vm storage. Have been doing this since 2012.
Can I limit the disk io bandwidth for a VM like I can in proxmox? This was a handy feature
Are you able to do Mac Monterey VM in Truenas Scale ? and if yes, how please ?
Would love more videos in truenas scale. Is it possible to go from core to scale without reinstall?
a series starting with a step by step, ground up build and then config, then onto virtualization would be great. I have always wanted to build a full sim of a house network in 1 physical box with everything. Router PFsense, windows and linux workststions, Servers, anyway just a thought
Same thing I was considering.
I need some help pls :( Just changed from Truenas core to scale and cant read ISOS for Vms, I get this error:
Error while creating the CDROM device. [EINVAL] attributes.path: 'libvirt-qemu' user cannot read from (Path) Please ensure correct permissions are specified
I gived all permissions trough the shell, any idea?? Ty
The next question is of course, can you run truenas as a VM in proxmox, that is run as a VM in truenas, that is run as a VM in proxmox?
Would really like to see you set up a plexmedia server with nzbget, radarr, sonarr etc on Trunas Scale VM.
I‘m currently planning my first home Server and I’m torn between truenas and unraid. What are your opinions on that? I wanna be able to easily expand my storage without spending $1000 on another 4 hard drives
if that is a priority, go Unraid, on TrueNAS you can't expand storage by just adding a single drive piecemeal
I've tried building VMs with TrueNAS Scale's built in hyperviser, and encountered these issues:
1.The VMs can see all the other machines on the same network except the TrueNAS host itself. I've seen posts on the TrueNAS forum suggesting making a bridge in TrueNAS network menu, but this didn't work for me.
2.I tried passing through an nvidia GPU to a windows 10 VM. Once I install nvidia's driver inside the VM, it becomes choppy as hell and incontrollable.
Avoid the unwanted commercial and jump to 1:10
Producing videos isn't cheap. If you enjoy the content, this is how I make money. 60 seconds cost you absolutely nothing from your day.
For me, the file server capabilities of trueNAS are much more important than the virtualization features and I don’t really want to deal with multiple boxes so I have a few VMs running, it works fine but I am looking forward to advancement in regards to vm management and features. The ability to run containers is fantastic though.
It worked fine but its a pain i endup got to configureing another box to operate truenas and using the old one for vm.
Hi! Is there possible to make Mac virtual machine in windows 10 with gpu passthrough? Maybe any chance with HyperV or Vmware?