Well that sorts my activity for this evening. Thank you for having a REAL walkthrough of this process where so many others miss the most common failures.
Jeff - At 5:22, you can usually just triple-click to select the whole line. Saves a bit of time instead of having to manually drag over the whole selection on a long command.
you do run into the issue of it picking up an unneeded space at the end of the line. Which may run you into some issues. When it comes to server reconfigs or anything infra related I like to take the time and make sure I'm using only what I need and spamming ctrl+c to make sure it's copied lol.
If you want to take the plex server to the limit, increase the RAM allotment and create a RAM disk to be used for transcoding output temp disk, it makes a huge difference
When passing through a multifunction I recomend you check the "All Fucntions" box when assigning the device. It will then show in the guest as one device with two or more functions (like it does on the host).
Thanks for this video. So clear and even in 2024 was followed perfectly. Added a Quadro P2000 to my ML350 Gen10 today and after setting up Proxmox and Plex over the weekend. I followed this today and it worked first time, no issues.
@@CraftComputing Say, any particular reason you didn't use an LXC container? I'm reading that GPU passthrough will still work in an LXC container, except that you can share the GPU with multiple containers instead of "marrying" it to a single VM using passthrough.
Coincidentally, I was doing this 3 days ago. I made VM, fiddled with proxmox to get passthrough working, and then installed Plex in the VM. But just then I've noticed that I need to have PlexPass in order to use encoding. And since I really don't want to pay for watching _absolutely 100% purchased and legally owned_ movies, I've scrapped Plex. In search for an alternative, I've found Jellyfin. It's opensource, totally free, UI looks 100x better (although sometimes you need to click twice on something), it isn't bloated with PlexTV crap and - it uses ffmpeg for encoding. That means you just basically install Jellyfin, nvidia drivers, ffmpeg, and change the default encoder in Jellyfin setting to ffmpeg. No need to register, purchase or subscribe to anything. It's excellent alternative to Plex, with every function that Plex have - but for free.
FYI for anyone having any mount issues. Not sure why, with my latest version of Ubuntu server to this date, I had to install samba, enable and then start it just before rebooting the vm to see the mount work correctly (at least back to my synology). Thanks for the video overall though! Really has helped me!
Thanks for this Jeff, finally have a machine dedicated to Proxmox that I can pass through an Nvidia card to. I've had this bookmarked since it came out and I'm really happy it worked for me! Keep up the great work!
Thank you for a very good guide. Small room for improvement: It would be worth mentioning that your CPU & Motherboard has to support IOMMU. Just like you covered what graphics cards support encoding/decoding what codec. I followed this guide only to find out in the last step that my Motherboard does not support IOMMU.
I have done this using Proxmox, Ubuntu Server VM, then Plex running in a Docker container, and with MakeMKV in another Docker container for automated ripping.
Many thanks for the excellent video and links / resources. I had been worried about moving my hypervisor from TrueNAS to Proxmox but your guidance has got me up and running. There are a few steps that myself (as a total rookie) had to adapt or side-google but that's probably a good thing as the accomplishment is even more satisfying. Can't thank you enough. All the best from NZ!
Just had to say Thanks! I'm a bit new to Proxmox, but this got me up and running with my Plex VM transcoding via onboard graphics. The bonus for me was the editing of the fstab file to auto-mount my shared storage. I could mount the drive from the command prompt, but this was gone after reboot. Thanks a ton for the crazy helpful content!
After watching for 18 months I've joined the Proxmox crew! Working on a new homelab server now. Found my old P1000 and enough parts to build a new toy and duplicate this video. Two servers should address my needs - Truenas - Video Minder - Plex - PiHole on first box.
I have recently followed this tutorial and it is still valid and works so thank you very much for such a clear tutorial. I did have one issue where the nvidia drivers would not load and I had to disable secure boot in order to get them to load correctly but they are now working perfectly. If anyone else has the same problem then simply press 'Esc' on when the Proxmox logo appears to enter the vm bios and disable secure boot
I just purchased a Quadro P400 to enable hardware encoding. My server is an old gaming PC based on an i5 3450. When I installed the P400, I couldn't add it to the Plex VM because the i5 3450 doesn't support VT-d. Luckily I found a cheap refurbished i5 3470 on Amazon. It now works like a charm. Thanks for the guide!
first: a real good guide to do this!!!!! second: my reccomendations to improve this guide (i am doing the same, but based on vmware free esxi) in /etc/fstab: add ",_netdev" after the password to ensure that the CIFS filesystem gets mounted after the network is full loaded. also a good improvement: add a dependency in systemd that the share should be mounted before plex starts. But this is definitly optional: currently plex handles a not accessible library very well!
You made my day, thanks for the tutourial. My plex server is now encoding with my P2200! It only took my 5 days fixing issues... your tutourial saved my ass. Thank you so much!!!
It amazes me that so many people are concerned for your sobriety or lack of. And they have the balls to ask. I think it would be obvious. The keyboard is so powerful and protective!
personally I don't understand the concern...he's not driving/flying/operating heavy equipment or doing anything life-critical, so...who gives a damn? Just a guy rambling in front of a camera while playing with his own stuff.
Dude, thank you for this video. I had issues getting HW transcoding to work ages ago, and forgot all about the P400 in my server. Thanks to your help, I've got it working again - many thanks
Thanks a lot, works like a charme. On Debian 10, you have also install "apt install linux-headers-$(uname -r) dkms build-essential" , to run the nvidia installer. Greetings from Germany
I’ve followed the channel for a good while now, and Im finally putting together what I consider to be my first “real” server, being a 3700x and 64 gigs of ram with a passed through Nvidia gpu. Hopefully I can utilize this well enough without breaking everything.
So um, when are you going to make a Part 8 for "Trying out SR-IOV - Cloud Gaming Server Part 7", since you've mistakenly assigned 1/16th of the GPU chip or 1/32th of GPU total power in that video?
I used to run plex in a VM just like you set it up in this video but found FAR superior performance when running it in an LXC container, still get full hardware support for my quandro p2000. It took some doing but the benefits are numerous.
Just to add to your video, it is important to verify if no other device is part of that IOMMU group, since it is not a group of devices but a group of psychical addresses that relate (also) to your PCI-E lanes any device inside that group also needs to be passed. Any one that got multiple devices (that are not part of the same card) inside one IOMMU group is running that device in a shared PCI-E slot, to change this said person would need to change that card to a dedicated slot in his motherboard. It is worth to note that most gpus wont use all the lanes assigned to them, so if you are planning on using more cards (say you want to run a dedicated Ethernet card + gpu) it may be on your best interest to purposefully use shared slots (to save pci-e lanes for other vms or expansion to the server for example).
My GPU ended up in the same IOMMU group as my HBA. I did not notice that until my storage crashed when vfio-pci took over for the adapter driver. I was able to add 'pcie_acs_override=downstream,multifunction' to my kernel command line to split the groups up further. After adding that it worked great.
@@lightmn To be fair it is not a guarantee of problems and there are workarounds like you just pointed out, but i would not recommend using a workaround unless you are sure that you can deal with problems that may pop up. Any way, you made a good call posting this, since some people reading may not have a dedicated pci-e slot remaining.
You should look into running the plex server as an LXC container. I have two running on Ubuntu 20.04 containers and sharing the same graphics card and network share. It lets me manage things a little better, with stellar performance!
why would you want to tie your whole hypervisor kernel to a potential panic caused by any LXC? if your system supports iommu and you run host level cpu in a raw format, the performance is almost the same
Thanks Jeff - glad you finally confirmed that VTd is possible on the Hyve Zeus. Based on your last video - I ordered the server and a pair of 2650s v2 procs. Wasn't going to use it for anything that would require PCI-E passthrough. But glad its there if I ever do.
I set mine up (about 1.5 years ago) using a linux container (LXC) on proxmox and shared the NVIDIA GPU with the LXC. It was much easier and less of a hassle than doing PCIe passthrough, though that's pretty easy these days if your hardware cooperates well enough. It's allowed some interesting flexibility, like being able to swap GPUs if you want to switch to something else since it seems to be tied to the PCIe slot and not to the device ID, which can change if you add or remove PCIe devices. Happy to provide more info if anyone is interested.
How did you pull that off, might I ask? nvidia-smi seems to work inside the lxc container, passed through properly. But then Plex will never (hw) transcode, all via CPU
great video. I recently bought a dell r620 for the purpose of running Plex and pfSense on it and actually couldn't find a tutorial that explained how to properly set up Proxmox for PCI-e pass-through. I'll see if i can get it working when i have a bit more time. ~EM
Great flavor on the video. Also you are so right on the beer this year. I would say it has more zesty note over the nutty acid from 2018 and 2019. I think the beer was good pairing with the video. The video like the beer requires some understanding to really be enjoyed.
I prefer to use the driverctl package to control the kernel drivers as I find it much cleaner and easier. As a bonus, if you have two identical cards (say HBAs) you can allow Proxmox to use one and pass through the other to a VM. You should also mention that the grub stuff does NOT apply to users who installed Proxmox with UEFI and ZFS root drives.
hey there, i just installed proxmox on zfs and i was going to go through this tutorial when i saw your comment. can you please point me into the right direction to make this work on zfs?
It’s probably better from a security standpoint to not put passwords exposed as plain text in the fstab file. Best to create a hidden credentials file with strict access permissions that can only be read/modified by root. Use the credentials file in the fstab instead to prevent your user credentials for your NAS from being exposed in the case of a breach. That way if the system was ever compromised they only have access to that system and not an easy pathway to access your data on the network.
For the poor fool who is on ZFS like myself and also wasted many hours trying to make this work: The kernel commandline needs to be placed as one line in /etc/kernel/cmdline. To apply your changes, run proxmox-boot-tool refresh, which sets it as the option line for all config files in loader/entries/proxmox-*.conf.
I bought the same server with the same components since I saw the video in which you showed it, now I'm going to try to put a graphics card on it and see if it works!
I found this video very helpful, even though I'm not using ProxMox. I've now got Hardware Encoding working with my Quadro P600 and Plex on Ubuntu Server 20.04 Thanks
Rebooting occasionally while making changes to important system files can help you figure out what you did wrong when your server doesn't boot properly.
For me, I had an issue of networking not being up yet when it tried to mount the network drives. I added a command to mount on access, which works great.
Hot Fix: When adding the GPU via PCIe device from the host. Especially if using a HP Proliant DL360 G7 as the host. You will need to select the GPU and ensure that the 'ROM Bar' box is checked. *Also on the host make sure your IOMMU and VFIO configs were saved. **Mine decided to show as saving, but didn't had to do it again. And, when you are hiding the Host CPU from the VM you will need to add the following line instead of modifying it: cpu: host,hidden=1 This is for Hiding the VM identifiers from Nvidia in the /etc/pve/qemu-server directory. Hopefully this helps others not have to spend hours troubleshooting like I did haha.
Thank you, Thank you, Thank you. the cpu: host,hidden=1 was the final step in getting my hw encoding to work on my setup. I originally thought it was due to my use of an AMD cpu without the intel extensions. This video finally got the last piece of the puzzle working.
Nice tutorial. I have an i7 3770k with 16gb RAM running win 10 as plex and file server. I'm not running Linux nor virtualization as I'm no comfortable with it yet. But your video sounds so easy that I will take a shot shot. I'm upgrading to a ryzen 5 3400g build, I'm feeling the pain of transferring everything from one server to another. If plex was already virtualized ir would be as simple as coping the image from one to another.
All I get when running nvidia-smi is the message: no devices were found. Do you have any idea how this could happen? I followed both your guide, the guide on Reddit and I googled the error, but they all tell me that I need to install the proprietary driver rather than the "open" one, but I am already using the proprietary driver!
Awesome video. It helped me a lot. I do have a question tho. Have you ever set up 2 p400s through proxmox? My current set up is 2 of them, 1 assigned to its own vm. I'm having issues with them being detected however. Would much change in the pve setup for adding in a second card?
Thank you very much for your video. May I have several questions cause I'm not sure if I understand all properly: 1. VGA should not be installed on host machine? 2. Does this setup work with only one GPU ? Will be Proxmox able to boot inside this "headless" mode and run all other VMs? 3. If I connect VGA output to monitor then until Proxmox start (after grub) I should see output from host machine, after Proxmox start VM which has GPU passtrought then the monitor will show output of this particular VM? 4. Can I just stop the VM (via web or ssh) which has passtrought GPU and run another VM with same passtrought GPU? 5. Is it possible or required to passtrought also USB for mouse and keyboard if I would like to have windows gaming VM? 6. I've seen various tutorials which require to use previously dumped & modified vBios of GPU, does Proxmox work without this? Many thanks
On your production instance are you mounting a share for the Plex DB? I'm slowly, multiplexing too many homelab projects burns time context switching :-(, attempting to stand up Plex/Jellyfin in a high availability fashion. I've been tempted to go Jellyfin/Kubernetes on some Jetsons but I'd also be interested in Plex on a Proxmox cluster if I could nail down a working incantation to stand up plex containerized and not have the Plex DB sh*t the bed during container migration. Or maybe a hybrid route with mundane plex instances farming out the transcodes ala kube-plex.... Add deciding which path to fumble down to the list of reasons completing projects seems glacial.
Hey Jeff, thanks for the tutorials, this one will be a great help over the holidays when I set up proxmox for the first time Question, I have a nvidia gpu that I was hoping to use for both plex and blueiris. I'm assuming the only way would be to run a Windows vm and have both applications running in there ( I.e. can't share / pass through to multiple vms)?
Awesome tutorial! I am doing Plex via Docker in Unraid along with Dockers for a VPN protected (PIA) Deluge client, Sonarr, Radarr, Tautulli, and DuckDNS/Nginx Proxy Manager for remote web access. I tried out Proxmox first but it just seemed a little too much for a simple NAS and media sharing setup.
So I noticed that your voice was without fan noise for most of the video.. besides the small clip you deliberately left it in.. Did you use RTX voice or something similar to remove it? If so that's genius..
Good Tutorial👍 Despite that I have a question: Should I use vlans in proxmox when creating VMs? I know what those are but is it worth it if like 3 VMs are communicating with each other because they need to (example Plex, Nas etc)? So they would be in the same Network with no vlan. Would that make Sense? For accessing the proxmox overlay I would like to get a second nic, so the VMs are not able to get to the proxmox overlay and its more secure?😅
TL:DR; of my headache, Proxmox hosts the Samba share from my ZFS pool, but a CT requires additional privilege to actually mount it. I learned the hard way, if you try to use a CT for Plex in Proxmox with Samba hosted on the same machine, it won't work. You will have to use a VM rather than the CT options. It seems like if you have Samba running on another machine a CT may work; but try this if you are bombarded by "Permission Denied" when trying to mount the CIFS.
Hey All, for the people that are having problems with nvidia-smi returning "no devices found". Install the Nvidia Driver version to what Jeff uses in the video. Solved issue for me.
Comment to the creator of this channel thank you for the insightful instructions on GPU passthrough this has been driving me crazy watching all these tutorials which don't work. your instructions truly made difference looking forward to more creative content and keep up the good work your doing for everyone.
Still not sure why you use PuTTY still. It's built into Windows 10 now. You can just ssh user@hostname now. I could see using it for serial connections, but I don't see you doing that much on the channel. Just something I have noticed since I subbed. Cheers!
Very good video very good explanation I liked it very much ....but I have one question for you, may I have the same setup but instead to run the Plex server in my own server could I use the Google Cloud platform? Will be the same or similar performance? or Plex needs to send the movie to the host(Google Cloud Platform) and then stream again to my home tv?
In the end I had to uninstall the nvidia driver I installed using this method thanks to UEFI bios restrictions. Then I followed some other directions and Installed Nvidia Drivers using a standard Ubuntu Repository. The driver install was very easy and nvidia-smi returns good data. It would be good to see a follow-up video on this subject...
This is a great tutorial, I'm looking forward to trying this out. However, I'm curious is there a way to run the GPU with a fan curve? I'm setting this up in an old workstation with a modified GPU (huge heatsink) that barely needs any fan assistance.
Please never tell people to install nvidia drivers that way. There are better ways to install the driver with apt-get that will keep it up to date automatically with the rest of the system. Linux is not Windows. Manually installing drivers from the web is a last resort.
Well that sorts my activity for this evening. Thank you for having a REAL walkthrough of this process where so many others miss the most common failures.
Nice work! Love NVENC for encoding/transcoding!
Oi hi tim good to see you here
@@frizzletits8511 can’t think of a better place!
The beer is cold, and so are the servers 😉
Hi Tim!
Nice to see you here!
I've been enjoying a similar setup with an old Quadro K4200 passed through! First time I've ever been ahead of your videos instead of following them!
Jeff - At 5:22, you can usually just triple-click to select the whole line. Saves a bit of time instead of having to manually drag over the whole selection on a long command.
you do run into the issue of it picking up an unneeded space at the end of the line. Which may run you into some issues. When it comes to server reconfigs or anything infra related I like to take the time and make sure I'm using only what I need and spamming ctrl+c to make sure it's copied lol.
If you want to take the plex server to the limit, increase the RAM allotment and create a RAM disk to be used for transcoding output temp disk, it makes a huge difference
When passing through a multifunction I recomend you check the "All Fucntions" box when assigning the device. It will then show in the guest as one device with two or more functions (like it does on the host).
I said the same thing lol
Thanks for this video. So clear and even in 2024 was followed perfectly. Added a Quadro P2000 to my ML350 Gen10 today and after setting up Proxmox and Plex over the weekend. I followed this today and it worked first time, no issues.
LITERALLY was about to do just this, and you release a video for it. You the effin' MAN!
I know
@@CraftComputing Say, any particular reason you didn't use an LXC container? I'm reading that GPU passthrough will still work in an LXC container, except that you can share the GPU with multiple containers instead of "marrying" it to a single VM using passthrough.
@@CraftComputing Have you considered using LXC rather than a whole VM?
I love these home server videos. Perfect for this platform.
Coincidentally, I was doing this 3 days ago. I made VM, fiddled with proxmox to get passthrough working, and then installed Plex in the VM. But just then I've noticed that I need to have PlexPass in order to use encoding. And since I really don't want to pay for watching _absolutely 100% purchased and legally owned_ movies, I've scrapped Plex. In search for an alternative, I've found Jellyfin. It's opensource, totally free, UI looks 100x better (although sometimes you need to click twice on something), it isn't bloated with PlexTV crap and - it uses ffmpeg for encoding. That means you just basically install Jellyfin, nvidia drivers, ffmpeg, and change the default encoder in Jellyfin setting to ffmpeg. No need to register, purchase or subscribe to anything. It's excellent alternative to Plex, with every function that Plex have - but for free.
Thanks for the excellent guide. Evidently two years later it's still gold!
FYI for anyone having any mount issues. Not sure why, with my latest version of Ubuntu server to this date, I had to install samba, enable and then start it just before rebooting the vm to see the mount work correctly (at least back to my synology). Thanks for the video overall though! Really has helped me!
Thanks for this Jeff, finally have a machine dedicated to Proxmox that I can pass through an Nvidia card to. I've had this bookmarked since it came out and I'm really happy it worked for me! Keep up the great work!
Thank you for a very good guide.
Small room for improvement:
It would be worth mentioning that your CPU & Motherboard has to support IOMMU. Just like you covered what graphics cards support encoding/decoding what codec.
I followed this guide only to find out in the last step that my Motherboard does not support IOMMU.
Yeah this turns out to be kind of important when you can't figure out why the GPU won't passthrough!
I have done this using Proxmox, Ubuntu Server VM, then Plex running in a Docker container, and with MakeMKV in another Docker container for automated ripping.
Many thanks for the excellent video and links / resources. I had been worried about moving my hypervisor from TrueNAS to Proxmox but your guidance has got me up and running. There are a few steps that myself (as a total rookie) had to adapt or side-google but that's probably a good thing as the accomplishment is even more satisfying. Can't thank you enough. All the best from NZ!
that, my friend, went smoothly. Proxmox 8.2.2, P400, Lubuntu, Plex - I thank you very much for this.
Just had to say Thanks! I'm a bit new to Proxmox, but this got me up and running with my Plex VM transcoding via onboard graphics. The bonus for me was the editing of the fstab file to auto-mount my shared storage. I could mount the drive from the command prompt, but this was gone after reboot. Thanks a ton for the crazy helpful content!
After watching for 18 months I've joined the Proxmox crew! Working on a new homelab server now. Found my old P1000 and enough parts to build a new toy and duplicate this video.
Two servers should address my needs - Truenas - Video Minder - Plex - PiHole on first box.
I have recently followed this tutorial and it is still valid and works so thank you very much for such a clear tutorial. I did have one issue where the nvidia drivers would not load and I had to disable secure boot in order to get them to load correctly but they are now working perfectly. If anyone else has the same problem then simply press 'Esc' on when the Proxmox logo appears to enter the vm bios and disable secure boot
I just purchased a Quadro P400 to enable hardware encoding. My server is an old gaming PC based on an i5 3450. When I installed the P400, I couldn't add it to the Plex VM because the i5 3450 doesn't support VT-d. Luckily I found a cheap refurbished i5 3470 on Amazon. It now works like a charm.
Thanks for the guide!
first: a real good guide to do this!!!!!
second: my reccomendations to improve this guide (i am doing the same, but based on vmware free esxi)
in /etc/fstab: add ",_netdev" after the password to ensure that the CIFS filesystem gets mounted after the network is full loaded.
also a good improvement: add a dependency in systemd that the share should be mounted before plex starts. But this is definitly optional: currently plex handles a not accessible library very well!
You made my day, thanks for the tutourial. My plex server is now encoding with my P2200! It only took my 5 days fixing issues... your tutourial saved my ass. Thank you so much!!!
It amazes me that so many people are concerned for your sobriety or lack of. And they have the balls to ask. I think it would be obvious. The keyboard is so powerful and protective!
personally I don't understand the concern...he's not driving/flying/operating heavy equipment or doing anything life-critical, so...who gives a damn? Just a guy rambling in front of a camera while playing with his own stuff.
Dude, thank you for this video. I had issues getting HW transcoding to work ages ago, and forgot all about the P400 in my server. Thanks to your help, I've got it working again - many thanks
every time i hear jeff talk about that lexar nvme drive at the beginning it makes me think he can be the price is right announcer
Considering the same setup, so your videos are really valuable - thank you for that!
Swear it wasn't the case last time I used plex
Thanks a lot, works like a charme. On Debian 10, you have also install "apt install linux-headers-$(uname -r) dkms build-essential" , to run the nvidia installer. Greetings from Germany
I’ve followed the channel for a good while now, and Im finally putting together what I consider to be my first “real” server, being a 3700x and 64 gigs of ram with a passed through Nvidia gpu. Hopefully I can utilize this well enough without breaking everything.
So um, when are you going to make a Part 8 for "Trying out SR-IOV - Cloud Gaming Server Part 7",
since you've mistakenly assigned 1/16th of the GPU chip or 1/32th of GPU total power in that video?
Thank you for this video. It bridged the gap between what I needed and what I knew which enabled me to be successful.
Impressive difference after you installed the card, thanks
This video teaches a lot more than Plex. Thanks!
Fantastic tutorial Jeff, and nothing went wrong too! How rare 🤣
I know, right? 😂😂😭
I used to run plex in a VM just like you set it up in this video but found FAR superior performance when running it in an LXC container, still get full hardware support for my quandro p2000. It took some doing but the benefits are numerous.
was it a privileged container?
@@dimitristsoutsouras2712 No, unprivileged. From what I understand that the only way hardware support will function currently.
Just to add to your video, it is important to verify if no other device is part of that IOMMU group, since it is not a group of devices but a group of psychical addresses that relate (also) to your PCI-E lanes any device inside that group also needs to be passed. Any one that got multiple devices (that are not part of the same card) inside one IOMMU group is running that device in a shared PCI-E slot, to change this said person would need to change that card to a dedicated slot in his motherboard. It is worth to note that most gpus wont use all the lanes assigned to them, so if you are planning on using more cards (say you want to run a dedicated Ethernet card + gpu) it may be on your best interest to purposefully use shared slots (to save pci-e lanes for other vms or expansion to the server for example).
My GPU ended up in the same IOMMU group as my HBA. I did not notice that until my storage crashed when vfio-pci took over for the adapter driver. I was able to add 'pcie_acs_override=downstream,multifunction' to my kernel command line to split the groups up further. After adding that it worked great.
@@lightmn To be fair it is not a guarantee of problems and there are workarounds like you just pointed out, but i would not recommend using a workaround unless you are sure that you can deal with problems that may pop up. Any way, you made a good call posting this, since some people reading may not have a dedicated pci-e slot remaining.
You should look into running the plex server as an LXC container. I have two running on Ubuntu 20.04 containers and sharing the same graphics card and network share. It lets me manage things a little better, with stellar performance!
why would you want to tie your whole hypervisor kernel to a potential panic caused by any LXC? if your system supports iommu and you run host level cpu in a raw format, the performance is almost the same
Thanks for this wonderful Proxmox video card pass through tutorial. Stay well!
Thanks Jeff - glad you finally confirmed that VTd is possible on the Hyve Zeus. Based on your last video - I ordered the server and a pair of 2650s v2 procs. Wasn't going to use it for anything that would require PCI-E passthrough. But glad its there if I ever do.
That walkthrough is gold.
I set mine up (about 1.5 years ago) using a linux container (LXC) on proxmox and shared the NVIDIA GPU with the LXC. It was much easier and less of a hassle than doing PCIe passthrough, though that's pretty easy these days if your hardware cooperates well enough. It's allowed some interesting flexibility, like being able to swap GPUs if you want to switch to something else since it seems to be tied to the PCIe slot and not to the device ID, which can change if you add or remove PCIe devices. Happy to provide more info if anyone is interested.
How did you pull that off, might I ask? nvidia-smi seems to work inside the lxc container, passed through properly. But then Plex will never (hw) transcode, all via CPU
great video. I recently bought a dell r620 for the purpose of running Plex and pfSense on it and actually couldn't find a tutorial that explained how to properly set up Proxmox for PCI-e pass-through. I'll see if i can get it working when i have a bit more time.
~EM
Great flavor on the video. Also you are so right on the beer this year. I would say it has more zesty note over the nutty acid from 2018 and 2019. I think the beer was good pairing with the video. The video like the beer requires some understanding to really be enjoyed.
You are a boss brother!
If this wasn't so niche I feel like you'd have millions of views.
I prefer to use the driverctl package to control the kernel drivers as I find it much cleaner and easier. As a bonus, if you have two identical cards (say HBAs) you can allow Proxmox to use one and pass through the other to a VM. You should also mention that the grub stuff does NOT apply to users who installed Proxmox with UEFI and ZFS root drives.
hey there, i just installed proxmox on zfs and i was going to go through this tutorial when i saw your comment. can you please point me into the right direction to make this work on zfs?
It’s probably better from a security standpoint to not put passwords exposed as plain text in the fstab file. Best to create a hidden credentials file with strict access permissions that can only be read/modified by root. Use the credentials file in the fstab instead to prevent your user credentials for your NAS from being exposed in the case of a breach. That way if the system was ever compromised they only have access to that system and not an easy pathway to access your data on the network.
hmm something I'll have to look into. Sounds very enticing.
13:10 I want to point out that if you use ZFS for your Proxmox install, you need to make these changes to systemd not grub as shown in the video.
Oh if I only read this comment earlier. Lots of googling before I found out this. And why wouldnt you use ZFS for your OS. Weird.
@@tydlig Was in the same boat, rewatched this video and thought. I am not going to be the only one x)
For the poor fool who is on ZFS like myself and also wasted many hours trying to make this work:
The kernel commandline needs to be placed as one line in /etc/kernel/cmdline. To apply your changes, run proxmox-boot-tool refresh, which sets it as the option line for all config files in loader/entries/proxmox-*.conf.
Thank you, this as a real walkthrough. Instead of reading a guide and possible making a mistake.
Excellent guide! Thank you so much! This really helped. I had to turn off secure boot in the Plex VM to install the GPU Driver,
Quardro P400 limits number of concurrent streams(sessions) to 3. Override those with keylase/nvidia-patch on github.
I was actually thinking of posting a quick video about unlocking the limiter.
A video about this would be greatly appreciated.
@@CraftComputing Oh yes please ! It will be very usefull ;)
I literally just spent the last two days figuring this shit out. My brain still hurts.
Perfect, thanks! This is one of my planned upgrades, so a nice tutorial should help.
I bought the same server with the same components since I saw the video in which you showed it, now I'm going to try to put a graphics card on it and see if it works!
I found this video very helpful, even though I'm not using ProxMox.
I've now got Hardware Encoding working with my Quadro P600 and Plex on Ubuntu Server 20.04
Thanks
really enjoyed the instructions as I will soon be running a Plex VM on my proxmox server, not so sure about crossing fingers or magic buttons.
Thank you for the info. have been working on this before, but now it works great
After doing some changes to fstab, it's faster to do mount -a instead a system reboot (;
Rebooting occasionally while making changes to important system files can help you figure out what you did wrong when your server doesn't boot properly.
@@beauregardslim1914 Exactly. I've had a mount -a work, but then the drive wouldn't mount on reboot for some reason.
For me, I had an issue of networking not being up yet when it tried to mount the network drives. I added a command to mount on access, which works great.
This is the video that i need !!!
These are an update video with docker/pod container for plex-nvidia ?
Thanks Jeff. It's like finishing Donkey Kong. Finally. Cheers,
Hot Fix:
When adding the GPU via PCIe device from the host. Especially if using a HP Proliant DL360 G7 as the host.
You will need to select the GPU and ensure that the 'ROM Bar' box is checked.
*Also on the host make sure your IOMMU and VFIO configs were saved. **Mine decided to show as saving, but didn't had to do it again.
And, when you are hiding the Host CPU from the VM you will need to add the following line instead of modifying it:
cpu: host,hidden=1
This is for Hiding the VM identifiers from Nvidia in the /etc/pve/qemu-server directory.
Hopefully this helps others not have to spend hours troubleshooting like I did haha.
Thanks for sharing this info. Will try it on my Proxmox. Just another question though: what't the type of keyboard you're using in this video ?
Thank you! Just what i need. Havent got it done previously. Going to try again after watching ur vid. Thanks.
Thanks for the great video! It helped me work through the details to pass my GPU through to my VM.
Thank you, Thank you, Thank you.
the cpu: host,hidden=1 was the final step in getting my hw encoding to work on my setup. I originally thought it was due to my use of an AMD cpu without the intel extensions. This video finally got the last piece of the puzzle working.
Anyone having the issue where if the VM reboots the NVIDIA drivers don't load? I have to reinstall the drivers each time.
Well done! Love your videos. Well put together and easy to follow. Keep up the great work!
That was very informative but surprisingly complicated. Not sure if I'd rather use a nvidia shield with a nas.
I'm so excited this worked! I watched the earlier video and was bummed for you. Lol
Nice tutorial. I have an i7 3770k with 16gb RAM running win 10 as plex and file server. I'm not running Linux nor virtualization as I'm no comfortable with it yet. But your video sounds so easy that I will take a shot shot. I'm upgrading to a ryzen 5 3400g build, I'm feeling the pain of transferring everything from one server to another. If plex was already virtualized ir would be as simple as coping the image from one to another.
All I get when running nvidia-smi is the message: no devices were found. Do you have any idea how this could happen? I followed both your guide, the guide on Reddit and I googled the error, but they all tell me that I need to install the proprietary driver rather than the "open" one, but I am already using the proprietary driver!
finally figured it out I needed to sh command in front of the driver to install- happy days ;)
Awesome video. It helped me a lot.
I do have a question tho.
Have you ever set up 2 p400s through proxmox? My current set up is 2 of them, 1 assigned to its own vm. I'm having issues with them being detected however. Would much change in the pve setup for adding in a second card?
I have a similar setup, just with VMware instead of Proxmox. Works great! Running a P600.
you made this a lot easier for me to do. thanks dood!
Thank you very much for your video. May I have several questions cause I'm not sure if I understand all properly:
1. VGA should not be installed on host machine?
2. Does this setup work with only one GPU ? Will be Proxmox able to boot inside this "headless" mode and run all other VMs?
3. If I connect VGA output to monitor then until Proxmox start (after grub) I should see output from host machine, after Proxmox start VM which has GPU passtrought then the monitor will show output of this particular VM?
4. Can I just stop the VM (via web or ssh) which has passtrought GPU and run another VM with same passtrought GPU?
5. Is it possible or required to passtrought also USB for mouse and keyboard if I would like to have windows gaming VM?
6. I've seen various tutorials which require to use previously dumped & modified vBios of GPU, does Proxmox work without this?
Many thanks
One day you will deploy a bind server on any of your servers so you can forget about remembering all the ips... good video, thanks!!!
For going near servers try ANC headphones/Earbuds, or something like Loops( hearing protection designed for concerts)
Thanks man! Worked like a charm
On your production instance are you mounting a share for the Plex DB?
I'm slowly, multiplexing too many homelab projects burns time context switching :-(, attempting to stand up Plex/Jellyfin in a high availability fashion. I've been tempted to go Jellyfin/Kubernetes on some Jetsons but I'd also be interested in Plex on a Proxmox cluster if I could nail down a working incantation to stand up plex containerized and not have the Plex DB sh*t the bed during container migration. Or maybe a hybrid route with mundane plex instances farming out the transcodes ala kube-plex.... Add deciding which path to fumble down to the list of reasons completing projects seems glacial.
Hey Jeff, thanks for the tutorials, this one will be a great help over the holidays when I set up proxmox for the first time
Question, I have a nvidia gpu that I was hoping to use for both plex and blueiris. I'm assuming the only way would be to run a Windows vm and have both applications running in there ( I.e. can't share / pass through to multiple vms)?
Youre making me want to give Proxmox another chance, cool video!
Thinking of sandboxing Pelx, nice video. What are your thoughts on RAM Disk for transcode temp folder, yes/no, size?
Awesome tutorial! I am doing Plex via Docker in Unraid along with Dockers for a VPN protected (PIA) Deluge client, Sonarr, Radarr, Tautulli, and DuckDNS/Nginx Proxy Manager for remote web access. I tried out Proxmox first but it just seemed a little too much for a simple NAS and media sharing setup.
So I noticed that your voice was without fan noise for most of the video.. besides the small clip you deliberately left it in.. Did you use RTX voice or something similar to remove it? If so that's genius..
Yep. RTX voice for the entire video.
@@CraftComputing that's so cool, I actually hadn't considered it's use in video production until seeing this.
Thanks for the vid! ✌
Good Tutorial👍
Despite that I have a question: Should I use vlans in proxmox when creating VMs? I know what those are but is it worth it if like 3 VMs are communicating with each other because they need to (example Plex, Nas etc)?
So they would be in the same Network with no vlan. Would that make Sense?
For accessing the proxmox overlay I would like to get a second nic, so the VMs are not able to get to the proxmox overlay and its more secure?😅
Excellent video As Always you're Jeff/Geoff Now realized that some hardware upgrades are wanted.
Awesome vid, as usual. How about doing this with a LXC container?
TL:DR; of my headache, Proxmox hosts the Samba share from my ZFS pool, but a CT requires additional privilege to actually mount it.
I learned the hard way, if you try to use a CT for Plex in Proxmox with Samba hosted on the same machine, it won't work. You will have to use a VM rather than the CT options.
It seems like if you have Samba running on another machine a CT may work; but try this if you are bombarded by "Permission Denied" when trying to mount the CIFS.
Hi, why use a VM instead of LXC? My Plex runs fine with LXC and I assume there's less overheads
New fan and this guide worked perfectly. Thank you so much!
New fan? Care to share?
Hey All, for the people that are having problems with nvidia-smi returning "no devices found". Install the Nvidia Driver version to what Jeff uses in the video. Solved issue for me.
thank you so much!
Comment to the creator of this channel thank you for the insightful instructions on GPU passthrough this has been driving me crazy watching all these tutorials which don't work. your
instructions truly made difference looking forward to more creative content and keep up the good work your doing for everyone.
Still not sure why you use PuTTY still. It's built into Windows 10 now. You can just ssh user@hostname now. I could see using it for serial connections, but I don't see you doing that much on the channel. Just something I have noticed since I subbed. Cheers!
came here to say this
Fantastic, thank you so much for the tutorial, very clear and helpful!
Very good video very good explanation I liked it very much ....but I have one question for you, may I have the same setup but instead to run the Plex server in my own server could I use the Google Cloud platform? Will be the same or similar performance? or Plex needs to send the movie to the host(Google Cloud Platform) and then stream again to my home tv?
In the end I had to uninstall the nvidia driver I installed using this method thanks to UEFI bios restrictions. Then I followed some other directions and Installed Nvidia Drivers using a standard Ubuntu Repository. The driver install was very easy and nvidia-smi returns good data. It would be good to see a follow-up video on this subject...
Is there a reason for using a VM instead of a CT?
Yeah i use a CT myself for this.
I'm pretty sure that version of the Nvidia driver is actually packaged for Ubuntu. I prefer to do that as it means someone else is keeping it patched.
This is a great tutorial, I'm looking forward to trying this out. However, I'm curious is there a way to run the GPU with a fan curve? I'm setting this up in an old workstation with a modified GPU (huge heatsink) that barely needs any fan assistance.
Please never tell people to install nvidia drivers that way. There are better ways to install the driver with apt-get that will keep it up to date automatically with the rest of the system. Linux is not Windows. Manually installing drivers from the web is a last resort.
@@SakuraChan00 Whatever is latest. The number is the driver version.