We're doing a VFIO PC giveaway!! Info here on Level1Linux + ruclips.net/video/HO_8liPirns/видео.html Enter on the forum! + forum.level1techs.com/t/vfio-2023-radeon-7000-edition-wip/199252 ~ Amber
Multi vm with one card is the single most desired feature for desktop gpus for me right now. I'd take a 50% performance cut at the same price just for sriov support on a consumer card. One day we'll get it I hope.
Same, except I want the performance and I'd like dynamic scaling to work so that I can overprovision. That way I can be doing work on AI or something for awhile then switch over to a gaming VM without having to stop the VM and reallocate the GPU.
I can't tell you how much I appreciate you carrying the VFIO torch. I've been playing with this forever but it's really hard to find updated info which is frustrating considering it is the future! It influenced the parts of my latest personal rig and I'm also spec-ing out a new forbidden everything box(maybe with a dash of llama?) and this video came out at the perfect time. I need to dig into the forums as I'm sure there's more info there but any additional info on AM4 options or other less (wallet)bleeding edge hardware would be swell.
Using a single GPU for multiple VMs would be excellent for hardware accelerated video encoding / decoding. This would be super useful when running CCTV and video transcoding on a home server. Such as Jellyfin and Frigate.
I agree. Even if I personally would prefer using multiple physical GPUs for different VMs, for lighter workloads such as video transcoding and other GPU accelerated tasks, it makes more sense to have one GPU running all of those home server type workloads. Although I'm still annoyed by the lack of PCIe expansion slots on a lot of newer motherboards. Even $300 AM5 motherboards will still only have 2 PCIe slots, with the second slot only being at 4x speed, not even 8x. And it's a shame that it's the Intel motherboards have this be somewhat more common since there you're giving up platform longevity that AM5 will (hopefully) destroy it in.
Depending on your comfort level, you can get a used H11SSL-i with a half decent Epyc for $350. MB comes with 3x16 and 3x8 PCIe 3.0's and fits into an ATX case. (don't forget the torque torx driver!)@@jamesbuckwas6575
Game DRM is what forced me to switch to a separate dedicated machine for gaming (I wasn't using VFIO but multiple GPU's passed through). I'd love to go back to virtualized but until that's sorted it's a tough sell.
Thanks for updating VFIO! I want to be able to share my GPU across many virtual machines similar to how you can share the nvidia GPU with docket containers. They share the same scheduler and have access to all the GPU memory
I'm mostly in the VFIO game for Hackintoshing, though I do have sandboxed Windows and Linux VMs. My ASRock 6800XT PG is working great with no reset issues. It would be absolutely fantastic if I could run multiple VMs simultaneously on the 6800XT -- it has plenty of resources to potentially run 2 VMs with decently high power for each, and even 4 VMs if GPU needs aren't terribly intensive.
I have built my vfio set up with proxmox as my host OS, then I have a Debian VM as my primary OS, and Win10 for gaming. I use a kvm to keep everything using the 4 monitors, and have recently used looking glass as well. Works flawlessly.
I had a crappy day, and watching this made me feel better. Wendell, you are Computer Bob Ross, and my blood pressure is lower because of it. Thank you!
I can't imagine having a system that can't do VFIO anymore. I'm doing it since 2021 and it has so many advantages that they're impossible to count. For example I'm not a fan of the whole TPM/secure boot thing that MS did with Windows 11. Well I do have a Win 11 gaming VM though. I'm just emulating the TPM 2.0 chip without actually using my hardware one. It enables me to be truly free in terms of software to the point that I dont have to care what big corps are pulling off. Oh you're requiring chip/feature XYZ now? Wait let me just slap it on my VM by choosing it from the list in virt-manager. In a day and age where our software and mainstream OSs are more and more ... externally controlled, this is really important. And you Wendell were my main inspiration to do all this back then, thank you.
Thank you!! Haven't been running vfio, but on my daily driver thanks to inspiration from you I've been running pci pass through running on poxmox. 3955WX in a super micro mobo. 6600 XT running arch as my primary desktop; P4000 on windows; P4000 on another debian box. And a P24 being used for some ML testing. All to different VMs.
Been running VFIO for years at this point. A VM for work, shut it down and then pass all the same hardware through to the gaming VM. Has it's niggles but the setup works for me. As others have said though, DRM is a bit of a kicker, some games even with a windows VM didn't work properly vs native windows.
Wendell I would like to start by thanking you so much for your time and dedication to VFIO. I couldn't agree more, this is the single biggest feature missing from modern day consumer hardware! The security features alone should being manufactured to release it, especially with the spotlight on cyber security in the last few years.
Excellent news. I really want to be able turn my 5950x/3090 into 3 PCs for use at LAN events. I figure 8GB of VRAM, 21GB of RAM and 10 threads per VM, that should be enough to run mid tier games at 1080p.
I was looking into setting up a VM with VFIO recently for gaming, but ended up dismissing the idea because of concerns about anti-cheat software. I've heard that some anti-cheat software will ban an account if it detects that it's running in a VM environment. This can supposedly be bypassed by spoofing hardware IDs, but that would look more suspicious IMO. Do you have any thoughts on this use case?
The only time I got PCI-E passthrough to work was in my forbidden router :) And even then I am really, really skeptical about missing something and accidentally spilling my LAN into the internet.
Having a second GPU compatible (or that I can be reasonably sure I can make work with my limited linux experience) with VFIO is the only thing I'm waiting to switch to linux. My ASUS Pro WS X570-ACE is just waiting there with its 128GB of ECC RAM wondering when I will install my R9 5950X in it. I wouldn't go NVidia as they are way too scummy to my taste (that sounds, wrong ...) but I was afraid that any GPU newer than my aging RX 5700XT would cause me too much issues for me to resolve down the line for me to comfortably go for VFIO. Thanks you Wendell for your work and dedication to that technology.
After years with the idea in the horizon but never seeing it viable to implement I was hoping to finally hop in with the performance the 7950X3D gives, this series could not have come at a better time! It's also great having AMD involved, I hope this will bring SR-VIO closer to the desktop!
I used to have a VFIO setup on my old R7 3800x system. I had a GT 1030 for the host and a Vega 56 that a passed through to the Windows VM with Ubuntu as the base system. Took a couple weeks to fully work out but was really fun. I'd love to do it again at some point with Arch this time and I've got a much better CPU to have some fun with (7800X3D)
This VFIO series could not have been timed better. I had my GPU die a couple weeks ago and picked up a 7600 to get the system back up and running. Did not really take the time to consider if it would handle VFIO properly and was basically just looking for something reasonably affordable. Eventually got the VM back up and running with passthrough, but I have been consistently plagued by reset issues on this particular card. Might also jump on the forums to try and sus out the cause of the reset issue :)
I want this specific setup for myself, and I want to run cables from the basement to the bedroom for monitor/keyboard/mouse/sound peripherals. Our bedroom gets SO HOT and LOUD and makes it hard to sleep. This would be the dream, it would also keep pet fur out of my machines...
Hi, I have an amd fx9590 cpu along with an amd rx 6600 and a gtx 1070 gpu. In proxmox i was able to vgpu unlock the gtx 1070 to make it have the sriov like features. i then use looking glass to capture 2 vm's using the gtx 1070 and display them on a linux vm using the rx 6600. i would like to show it off on the level1tech forum or on youtube for fun.
Two years ago I managed to get a single Xeon 14core something and 2 GTX1060s to run like two physical PCs. Getting the two GPUs dedicated each to their own VMware instance was difficult, but getting USB devices reliably assigned to their dedicated VM was a pain. PAIN!!! Getting more than one USB sound source to work was impossible. Fortunately one of my displays supported HDMI sound, which worked flawlessly. Thanks nVidia. Interesting project to work on, but scrapped it half a year later due to USB constantly pissing me off.
Just did an ITX vfio build with a 7700x and using the iGPU on the host. The biggest landmine I found was that kernels 6.2 and 6.3 didnt support display port 2.0 with the built in ryzen 7000 igpu. I was only able to output 4k 90hz on a 144hz panel. My solution was to just stick with debian bookworm
Interesting mobo choice, I was considering it but ended up with the MSI MEG ACE - I feel like it has better separation of I/O on the ports. It also has one USB-C controller on an isolated pcie port, which allows me to passthrough the whole port. 10g Ethernet controller allows for no-compromise in speeds when doing the virt nic. While SATA controller is also on the passthrough list - I should probably share the config on l1forums. There are some quirks - running an older bios. In theory, it has the and patch and the new memory support so I can run 5600mhz with 192gb (yup, it works!). Your Linux GPU has to be on the first slot (or use the usb-c GPU, but I had a few minor glitches, besides it wasn't doing 4k).
I like the concept of making spyware obsolete by making telemetry pointless. You even could randomize application virtual hardware resources a little to fool any tracking software.
I was actualy thinking around for a similar project and was looking for states of the art (I try every 2-3 years). Glad I can read fresh and proper how to. I will try to also use vGPU since I am still on Turing line. I wonder if it is possible to configure both a Hook and a vGPU and choose between vGPU or hook when starting the VM depending on what we want to do in it ?
And I can set it up in under 30 minutes, with a few automated scripts for near bare metal performance. it really has become much easier to just get up and running
AMD what happened to MxGPU, RocM and of course SRIOV? Shouldn’t that become the(!) way to compartmentalize, share (“consumer”) GPUs and secure your system, without having to buy server/workstation GPU? Thanks to Wendell the Great aka “PC janitor” (of the century) and @Level1Techs for this wonderful giveaway and good luck to all who can be part of it. Cheers out of the old world of Europe, Pepe
I do this every day for years now, with a 9900K, iGPU for the host (Ubuntu), a GTX1660 for Win10 vm, and a GT710 for a Linux vm, this connected to a crt display (for emulators and old stuff). I pin the second thread of each core to the vms (6 to the win10 one and 2 to the linux one, in a reverse order, cpu #15 is the cpu #0 of the win10 vm), and leave the first thread to the host. Mitigations off. I got no problems with antipiracy, but I use only single player games and I use the hidden kvm option (tested GTA V, RDR2, ROTTR, SOTTR, AC Origins, Death Stranding, Flight Simulator, Hitman, Alien Isolation and others I can't remember).
Wendell, greetings from CHIAPAS México. I have a question, can I enter the giveaway even though I've never used VFIO in my life? Amber the thumbnail and editing o this video, you nailed it.
You know what would be really cool? Vfio passthrough with memory sharing via memballoon or a similar mechanism. If your guest VM doesn't use all its memory it's so wasteful to allocate it the entire range and not give it back to the host.
Great timing Wendell! Just picked up a second gpu and VFIO is top of my list for things to mess around with. Never had the benefit of 2 working gpus before to try it properly.
I want to follow this but I dont have and can't afford the required hardware. Gonna watch anyways and I REALLY hope I win the setup so I can follow along!
My knowledge with this sort of thing is very entry level. What would be the main difference between this setup and just running TrueNas as host and separating the cores and assigning GPUs to each instance that way?
This is such valuable content. Am looking into doing this to use Windows only hardware for music production, if possible, and dual booting is just not a viable option.
In general it should work. What I've learned, is that you shouldnt passthrough the device directly unless it is a pci device. If it's a usb device pass through the usb controller etc. That way I've managed to get blu ray drm to work.
I've been dreaming a similar setup (but based on a threadripper instead) for quite a while, but it'll have to wait since single-gpu passthrough can be pretty iffy…
I've been using VMs on windows with VMware just because it has support for 3d acceleration with Nvidia cards. Would like to use KVM based VMs but needing separate GPUs for VMs makes no sense for my use case. I thought this was showcasing virtualizing a single GPU for 2 VMs but looks like you're just using PCIE-Passthrough and 2 GPUs?
You think with this system (7950x3d and same taichi x670e mb etc) that the igpu from the cpu can be passedthrough? I have been trying for about 2 weeks now and 0 progress what so ever. the standard PCI card works fine, just not the igpu, to the point where now I just want to know if it is even possible so I don't waste more days on this.
Would love to be able to spinup a windows 7 instance to get my itens on the japanese lolpso2 server for some nostalgia. Sure as heck not going to dual boot to do it. Sure as heck doesn't work in limmux.
It's STUPID that Nvidia puts their GRID driver behind a dev-wall (possibly a pay wall?). I would LOVE to pick up an older multi-GPU Nvidia card, and split the passthrough to different VMs (which Craft Computing has already demonstrated). My Proxmox server doesn't have the physical room to install multiple almost 3-slot GPUs, but it would be nice to be able to passthrough different GPUs to different VMs.
Would it be possible to get VFIO to work with Intel igpu for Linux and then a powerful GPU for gaming on windows? I'd like to try tinkering with it on a SFF PC
I'm right there with you, hopefully more people respond. I have a decent nuc type system with an Intel gpu I want to use for transcoding and what not for jellyfin.
@@rudysal1429 Tons of people have igpus I think it would be a great way to popularize VFIO and Linux since it would be easier to have a system that has the benefits of both windows and Linux
Running games in a vm do not introduce more input lag and display lag? There are some games that require low input/display lag ; mainly fighting games 😮💨
You mention putting DRM and Anti-Cheat in a VM to get it away from your main system but in my experience (after about 6 months of daily driving a gaming VM) anti-cheat is getting more and more restrictive over where it will even run and, seemingly more often than not these days, completely forbids you from launching games in VM environments. That's what finally got me to go back to my older bare-metal system. 70% of the games I wanted to play wouldn't work in a VM because of anti-cheat. And I swear that 70% figure was lower at the beginning of that 6 month period so things are getting worse. I'm sure you probably know something that I don't, so what's the workaround for this issue?
I really hope AMD makes ultra cheap GPU splitting. I run a bunch of servers which need GPUs and wish they’d just make this easy. I run 3 machines via pcie pass through for dev and really would like 4 or 8! I have a couple epyc systems and ryzen and my pcie pass through system is currently running proxmox. Its very good for cicd builders and will never spend the cash on an instinct card, it should be part of the consumer cards!
Work wont let me run mixed VMs, some with GPU acceleration and some without. I have to have a separate host with dedicated VDI. Made me sad. Maybe I'm just thick?
Can work, but you might get some problems with the 5000 series RX. Some have the reset bug, some don't. IOMMU as a bios feature has been around since a decade, so chances are that you mainboard supports it in general.
Tbh Ive never had good luck with AMD cards, two Polaris and one Vega failed to work, even with vendor-reset and all bells and whistles. In contrast, Nvidia just works since they explicitly allow passthrough for consumer cards in their driver. I've had a 1660 Super and now a 3060ti and I didnt have to patch kernels, fiddle with ROMs, drivers etc., it just always worked. @@AlfieMakes
![[demo] VFIO in 2024 // windows + linux](http://i.ytimg.com/vi/6SoteC1FM14/mqdefault.jpg)
We're doing a VFIO PC giveaway!!
Info here on Level1Linux
+ ruclips.net/video/HO_8liPirns/видео.html
Enter on the forum!
+ forum.level1techs.com/t/vfio-2023-radeon-7000-edition-wip/199252
~ Amber
Seggs with Wendell
Fully penetrative seggs with wendell
Multi vm with one card is the single most desired feature for desktop gpus for me right now. I'd take a 50% performance cut at the same price just for sriov support on a consumer card. One day we'll get it I hope.
step one, buy a down to 10 series nvidia GPU, step two, run vgpu unlocker, done
@@malloot9224 driver?
@@Infinitex9 what do you mean?
pReach! :D
Same, except I want the performance and I'd like dynamic scaling to work so that I can overprovision.
That way I can be doing work on AI or something for awhile then switch over to a gaming VM without having to stop the VM and reallocate the GPU.
THIS is what I come to your channel for. NOBODY else can match it.
mad props to the editor for that thumbnail, holy crap 😂
Super cool
That would be a Grant special ;P ~ Amber
I can't tell you how much I appreciate you carrying the VFIO torch. I've been playing with this forever but it's really hard to find updated info which is frustrating considering it is the future! It influenced the parts of my latest personal rig and I'm also spec-ing out a new forbidden everything box(maybe with a dash of llama?) and this video came out at the perfect time.
I need to dig into the forums as I'm sure there's more info there but any additional info on AM4 options or other less (wallet)bleeding edge hardware would be swell.
Thank you so much for all the work you do for this!
Using a single GPU for multiple VMs would be excellent for hardware accelerated video encoding / decoding. This would be super useful when running CCTV and video transcoding on a home server. Such as Jellyfin and Frigate.
I agree. Even if I personally would prefer using multiple physical GPUs for different VMs, for lighter workloads such as video transcoding and other GPU accelerated tasks, it makes more sense to have one GPU running all of those home server type workloads.
Although I'm still annoyed by the lack of PCIe expansion slots on a lot of newer motherboards. Even $300 AM5 motherboards will still only have 2 PCIe slots, with the second slot only being at 4x speed, not even 8x. And it's a shame that it's the Intel motherboards have this be somewhat more common since there you're giving up platform longevity that AM5 will (hopefully) destroy it in.
Depending on your comfort level, you can get a used H11SSL-i with a half decent Epyc for $350. MB comes with 3x16 and 3x8 PCIe 3.0's and fits into an ATX case. (don't forget the torque torx driver!)@@jamesbuckwas6575
All of those applications can run on a docker container with GPU acceleration
Game DRM is what forced me to switch to a separate dedicated machine for gaming (I wasn't using VFIO but multiple GPU's passed through). I'd love to go back to virtualized but until that's sorted it's a tough sell.
What games didn’t work?
Anti cheat systems also don't like virtualization. I can kind of understand why, but it prevents you from playing most multiplayer games.
@@Tuxm. At the time it was R6 Siege but anything that uses BattleEye doesn't currently allow virtualization
@@r0ckr87 respectfully your doing it wrong only games that don’t work are ones with kernel level anti cheat which is few so far
@@Tuxm. True, it's mostly high level competitive games that do this. Some DRM too I think?
Amazing what you are doing please dont ever stop this VFIO work!
Thanks for updating VFIO! I want to be able to share my GPU across many virtual machines similar to how you can share the nvidia GPU with docket containers. They share the same scheduler and have access to all the GPU memory
This is golden content. Hope you push through with this. And I agree that this is the future.
I'm mostly in the VFIO game for Hackintoshing, though I do have sandboxed Windows and Linux VMs. My ASRock 6800XT PG is working great with no reset issues.
It would be absolutely fantastic if I could run multiple VMs simultaneously on the 6800XT -- it has plenty of resources to potentially run 2 VMs with decently high power for each, and even 4 VMs if GPU needs aren't terribly intensive.
I have built my vfio set up with proxmox as my host OS, then I have a Debian VM as my primary OS, and Win10 for gaming. I use a kvm to keep everything using the 4 monitors, and have recently used looking glass as well. Works flawlessly.
I had a crappy day, and watching this made me feel better. Wendell, you are Computer Bob Ross, and my blood pressure is lower because of it. Thank you!
Always great to see a video that's not the weekly show. Although I do love links with friends too!
Thankful for your channel. Top quality content and its always fun to watch.
I can't imagine having a system that can't do VFIO anymore. I'm doing it since 2021 and it has so many advantages that they're impossible to count. For example I'm not a fan of the whole TPM/secure boot thing that MS did with Windows 11. Well I do have a Win 11 gaming VM though. I'm just emulating the TPM 2.0 chip without actually using my hardware one. It enables me to be truly free in terms of software to the point that I dont have to care what big corps are pulling off. Oh you're requiring chip/feature XYZ now? Wait let me just slap it on my VM by choosing it from the list in virt-manager. In a day and age where our software and mainstream OSs are more and more ... externally controlled, this is really important.
And you Wendell were my main inspiration to do all this back then, thank you.
Thank you!! Haven't been running vfio, but on my daily driver thanks to inspiration from you I've been running pci pass through running on poxmox. 3955WX in a super micro mobo. 6600 XT running arch as my primary desktop; P4000 on windows; P4000 on another debian box. And a P24 being used for some ML testing. All to different VMs.
Spot on, I have that same wood handle screwdriver at mark 0:57.
Thx for all you do for the community
This is just... the coolest thing. I cant even express how awesome this is.
This is the content I've been waiting for.
Thank you for your work, This will be epic one day!
Been running VFIO for years at this point. A VM for work, shut it down and then pass all the same hardware through to the gaming VM. Has it's niggles but the setup works for me. As others have said though, DRM is a bit of a kicker, some games even with a windows VM didn't work properly vs native windows.
Is it hard to run this vfio on something like a Intel nuc? Wanting to try proxmox on an extra nuc I have
@@rudysal1429 depends on which model and if it supports vt-d.
Wendell I would like to start by thanking you so much for your time and dedication to VFIO. I couldn't agree more, this is the single biggest feature missing from modern day consumer hardware! The security features alone should being manufactured to release it, especially with the spotlight on cyber security in the last few years.
super cool. this is a dream build for passthrough
This was a great video.
It's got me looking into building another machine again. Winning the Giveaway would be amazing thought, LOL.
Excellent news. I really want to be able turn my 5950x/3090 into 3 PCs for use at LAN events. I figure 8GB of VRAM, 21GB of RAM and 10 threads per VM, that should be enough to run mid tier games at 1080p.
That case is HUGE! and I want it!!! Lol
I was looking into setting up a VM with VFIO recently for gaming, but ended up dismissing the idea because of concerns about anti-cheat software. I've heard that some anti-cheat software will ban an account if it detects that it's running in a VM environment. This can supposedly be bypassed by spoofing hardware IDs, but that would look more suspicious IMO. Do you have any thoughts on this use case?
Wow, just a day after I left a comment asking for an update on what's currently the best VFIO build!
0:27 love the startrek reference
foshizzle -Wendell
Sounds good :D
Thank you. Vm multi GPU instant for 2 gpus... I had more enjoyment with vista usage. Then it getting the most basic vm usage
The only time I got PCI-E passthrough to work was in my forbidden router :) And even then I am really, really skeptical about missing something and accidentally spilling my LAN into the internet.
Having a second GPU compatible (or that I can be reasonably sure I can make work with my limited linux experience) with VFIO is the only thing I'm waiting to switch to linux.
My ASUS Pro WS X570-ACE is just waiting there with its 128GB of ECC RAM wondering when I will install my R9 5950X in it.
I wouldn't go NVidia as they are way too scummy to my taste (that sounds, wrong ...) but I was afraid that any GPU newer than my aging RX 5700XT would cause me too much issues for me to resolve down the line for me to comfortably go for VFIO.
Thanks you Wendell for your work and dedication to that technology.
Hell yea!
Me: Maybe I'll pass on Ryzen 7000.
Wendell: Look at this VFIO build!
Me: I'm listening...
After years with the idea in the horizon but never seeing it viable to implement I was hoping to finally hop in with the performance the 7950X3D gives, this series could not have come at a better time!
It's also great having AMD involved, I hope this will bring SR-VIO closer to the desktop!
I used to have a VFIO setup on my old R7 3800x system. I had a GT 1030 for the host and a Vega 56 that a passed through to the Windows VM with Ubuntu as the base system. Took a couple weeks to fully work out but was really fun. I'd love to do it again at some point with Arch this time and I've got a much better CPU to have some fun with (7800X3D)
that thumbnail! 10/10
10/10 for the thumbnail. 😅
This is definitely Level Two content...😏
I love that thumbnail more than I should
I'm running freebsd as my host. No fix for Navi reset bug, but other than that, managing VMs with first class zfs support has been incredible.
This VFIO series could not have been timed better.
I had my GPU die a couple weeks ago and picked up a 7600 to get the system back up and running. Did not really take the time to consider if it would handle VFIO properly and was basically just looking for something reasonably affordable. Eventually got the VM back up and running with passthrough, but I have been consistently plagued by reset issues on this particular card.
Might also jump on the forums to try and sus out the cause of the reset issue :)
I want this specific setup for myself, and I want to run cables from the basement to the bedroom for monitor/keyboard/mouse/sound peripherals. Our bedroom gets SO HOT and LOUD and makes it hard to sleep. This would be the dream, it would also keep pet fur out of my machines...
Love the thumbnail 😂
(Here's one for the algorithm) Thank you, Wendell...🇺🇸 😎👍☕
Great!
Hi, I have an amd fx9590 cpu along with an amd rx 6600 and a gtx 1070 gpu. In proxmox i was able to vgpu unlock the gtx 1070 to make it have the sriov like features. i then use looking glass to capture 2 vm's using the gtx 1070 and display them on a linux vm using the rx 6600. i would like to show it off on the level1tech forum or on youtube for fun.
Two years ago I managed to get a single Xeon 14core something and 2 GTX1060s to run like two physical PCs. Getting the two GPUs dedicated each to their own VMware instance was difficult, but getting USB devices reliably assigned to their dedicated VM was a pain. PAIN!!! Getting more than one USB sound source to work was impossible. Fortunately one of my displays supported HDMI sound, which worked flawlessly. Thanks nVidia. Interesting project to work on, but scrapped it half a year later due to USB constantly pissing me off.
*dozens of us*! Yep, maybe. But what a dedicated bunch.
Did Wendell just quote Arrested Development!! Lmao
I feel like a starving man looking through a restaurant window with my decade old GPU and this video.
Just did an ITX vfio build with a 7700x and using the iGPU on the host. The biggest landmine I found was that kernels 6.2 and 6.3 didnt support display port 2.0 with the built in ryzen 7000 igpu. I was only able to output 4k 90hz on a 144hz panel. My solution was to just stick with debian bookworm
Interesting mobo choice, I was considering it but ended up with the MSI MEG ACE - I feel like it has better separation of I/O on the ports. It also has one USB-C controller on an isolated pcie port, which allows me to passthrough the whole port.
10g Ethernet controller allows for no-compromise in speeds when doing the virt nic.
While SATA controller is also on the passthrough list - I should probably share the config on l1forums.
There are some quirks - running an older bios. In theory, it has the and patch and the new memory support so I can run 5600mhz with 192gb (yup, it works!).
Your Linux GPU has to be on the first slot (or use the usb-c GPU, but I had a few minor glitches, besides it wasn't doing 4k).
After that GN Kiddy Drama good Content is so refreshing to see =D Very Interresting
I like the concept of making spyware obsolete by making telemetry pointless. You even could randomize application virtual hardware resources a little to fool any tracking software.
I was actualy thinking around for a similar project and was looking for states of the art (I try every 2-3 years).
Glad I can read fresh and proper how to.
I will try to also use vGPU since I am still on Turing line.
I wonder if it is possible to configure both a Hook and a vGPU and choose between vGPU or hook when starting the VM depending on what we want to do in it ?
And I can set it up in under 30 minutes, with a few automated scripts for near bare metal performance. it really has become much easier to just get up and running
0:20 I cannot agree more....
AMD what happened to MxGPU, RocM and of course SRIOV? Shouldn’t that become the(!) way to compartmentalize, share (“consumer”) GPUs and secure your system, without having to buy server/workstation GPU?
Thanks to Wendell the Great aka “PC janitor” (of the century) and @Level1Techs for this wonderful giveaway and good luck to all who can be part of it.
Cheers out of the old world of Europe,
Pepe
I do this every day for years now, with a 9900K, iGPU for the host (Ubuntu), a GTX1660 for Win10 vm, and a GT710 for a Linux vm, this connected to a crt display (for emulators and old stuff). I pin the second thread of each core to the vms (6 to the win10 one and 2 to the linux one, in a reverse order, cpu #15 is the cpu #0 of the win10 vm), and leave the first thread to the host. Mitigations off. I got no problems with antipiracy, but I use only single player games and I use the hidden kvm option (tested GTA V, RDR2, ROTTR, SOTTR, AC Origins, Death Stranding, Flight Simulator, Hitman, Alien Isolation and others I can't remember).
Perhaps place a link to the "guide on the level1 forums" in the video description.
Wendell, greetings from CHIAPAS México.
I have a question, can I enter the giveaway even though I've never used VFIO in my life?
Amber the thumbnail and editing o this video, you nailed it.
Sure
@@Level1Techs Thank you for answering. Greetings from Chiapas México.
You know what would be really cool? Vfio passthrough with memory sharing via memballoon or a similar mechanism. If your guest VM doesn't use all its memory it's so wasteful to allocate it the entire range and not give it back to the host.
Virtual Function IO Passthrough, with a giveaway! Would you ship that to Australia?
0:32 well of course due to exploring the galaxy and all that
Great timing Wendell! Just picked up a second gpu and VFIO is top of my list for things to mess around with. Never had the benefit of 2 working gpus before to try it properly.
Have y'all thought of different costumes to do for October?
I want to follow this but I dont have and can't afford the required hardware. Gonna watch anyways and I REALLY hope I win the setup so I can follow along!
My knowledge with this sort of thing is very entry level. What would be the main difference between this setup and just running TrueNas as host and separating the cores and assigning GPUs to each instance that way?
I'm guessing TrueNas doesn't let you use the one GPU for two OS at the same time.
I don't have any AMD gpus but I have an Intel ARC that's been waiting for me to get a bit of time to set this up.
I had pass through setup with gen 1 thread ripper and two 1070s. One bios update and boom I’ve never been able to get it to work properly since.
this is the holy grail / ideal setup in my mind, but it's such an edge case in the world :(
NICE!
Need a new cpu now :)
This is such valuable content. Am looking into doing this to use Windows only hardware for music production, if possible, and dual booting is just not a viable option.
We're in the same boat!
In general it should work. What I've learned, is that you shouldnt passthrough the device directly unless it is a pci device. If it's a usb device pass through the usb controller etc. That way I've managed to get blu ray drm to work.
Cool
I've been dreaming a similar setup (but based on a threadripper instead) for quite a while, but it'll have to wait since single-gpu passthrough can be pretty iffy…
I've been using VMs on windows with VMware just because it has support for 3d acceleration with Nvidia cards. Would like to use KVM based VMs but needing separate GPUs for VMs makes no sense for my use case. I thought this was showcasing virtualizing a single GPU for 2 VMs but looks like you're just using PCIE-Passthrough and 2 GPUs?
LOL, that Eminem look!!!
Been running for years, even updating the guides. Unfortunately not on the latest HW.
You think with this system (7950x3d and same taichi x670e mb etc) that the igpu from the cpu can be passedthrough? I have been trying for about 2 weeks now and 0 progress what so ever. the standard PCI card works fine, just not the igpu, to the point where now I just want to know if it is even possible so I don't waste more days on this.
Would love to be able to spinup a windows 7 instance to get my itens on the japanese lolpso2 server for some nostalgia.
Sure as heck not going to dual boot to do it. Sure as heck doesn't work in limmux.
dunno wtf a vfio is, but that thumbnail got my click. guess it's time to learn.
😂 I feel the same, that Eminem edit is WILD!
That thumbnail got me too 😂
I've become a real fan of Asrock recently. But then, I'm a homelabber, not a gamer.
It's STUPID that Nvidia puts their GRID driver behind a dev-wall (possibly a pay wall?).
I would LOVE to pick up an older multi-GPU Nvidia card, and split the passthrough to different VMs (which Craft Computing has already demonstrated).
My Proxmox server doesn't have the physical room to install multiple almost 3-slot GPUs, but it would be nice to be able to passthrough different GPUs to different VMs.
Would it be possible to get VFIO to work with Intel igpu for Linux and then a powerful GPU for gaming on windows? I'd like to try tinkering with it on a SFF PC
I'm right there with you, hopefully more people respond. I have a decent nuc type system with an Intel gpu I want to use for transcoding and what not for jellyfin.
@@rudysal1429 Tons of people have igpus I think it would be a great way to popularize VFIO and Linux since it would be easier to have a system that has the benefits of both windows and Linux
You can use Intel iGPU for Linux host, and pass the pcie GPU to a vm (linux or windows or whatever you want)
Running games in a vm do not introduce more input lag and display lag? There are some games that require low input/display lag ; mainly fighting games 😮💨
AsRock is still putting sticker on the memory dimm slots.😅
Mmmmm specialized content
I wanna just spin up a Windows VM on my xcp-ng host and play a game every now and then.
You mention putting DRM and Anti-Cheat in a VM to get it away from your main system but in my experience (after about 6 months of daily driving a gaming VM) anti-cheat is getting more and more restrictive over where it will even run and, seemingly more often than not these days, completely forbids you from launching games in VM environments. That's what finally got me to go back to my older bare-metal system. 70% of the games I wanted to play wouldn't work in a VM because of anti-cheat. And I swear that 70% figure was lower at the beginning of that 6 month period so things are getting worse. I'm sure you probably know something that I don't, so what's the workaround for this issue?
Curious about how 256GB or 192GB DDR5 on AM5 is going with the recent BIOS updates
Linus had made a video few months back about some Win 11 feature which allows us to share the same gpu to vm's. Has anyone tried that?
I really hope AMD makes ultra cheap GPU splitting. I run a bunch of servers which need GPUs and wish they’d just make this easy. I run 3 machines via pcie pass through for dev and really would like 4 or 8! I have a couple epyc systems and ryzen and my pcie pass through system is currently running proxmox. Its very good for cicd builders and will never spend the cash on an instinct card, it should be part of the consumer cards!
Work wont let me run mixed VMs, some with GPU acceleration and some without. I have to have a separate host with dedicated VDI. Made me sad. Maybe I'm just thick?
Can I try this on AM4?.
Maybe the 5700G paired with a 5000 series card? (RX 5700XT comes to mind)
Can work, but you might get some problems with the 5000 series RX. Some have the reset bug, some don't. IOMMU as a bios feature has been around since a decade, so chances are that you mainboard supports it in general.
@@Spicysauced I've got an ASRock B450 Steel Series. I'll give it a try, otherwise I'll try to get a 6000 series GPU then
Tbh Ive never had good luck with AMD cards, two Polaris and one Vega failed to work, even with vendor-reset and all bells and whistles. In contrast, Nvidia just works since they explicitly allow passthrough for consumer cards in their driver. I've had a 1660 Super and now a 3060ti and I didnt have to patch kernels, fiddle with ROMs, drivers etc., it just always worked. @@AlfieMakes