Me too. It's pretty much the dream I've had for 30+ years. We've had the hardware to do what I want for a while, we just needed the software not to be hidden behind massive paywalls.
Just wanted to give my heartfelt thanks to you Jeff. A while back I discovered your channel and have binge-watched most of your videos regarding cloud gaming, virtualisation and server hardware since then. Your videos (especially the Hyper-V paravirtualisation) allowed me to get my sort-of gaming-server (Im still sitting on it bare-metal but also hosting VMs and streaming all over the place) using Hyper-V GPU Paravirtualisation going, something I wanted to do for over a decade. 2 simultaneous games? No problem. Streaming an emulator running in a vm to a computer 300km away? Works just fine. Incidentially this has also revitalised some old laptops that can now be given some computing power. Ill be testing out 3x Baldurs Gate on one 6900XT + 7900X3D this weekend, lets hope it works. If it does 2 people will be saving quite some money not having to buy a computer.
Yes, please cover virtualization on consumer Ampere, that would be sick! The RTX 3090 is typically hundreds of dollars cheaper on the used market than its A-series VRAM equivalents, but with even more cores!
@@pachete. That’s not strictly true; LTT posted a video of them splitting a 3080ti into two VMs two years ago using Windows Hyper-V. I know that isn’t as appealing as just running the VM’s through a more lightweight OS (or frankly, any OS that isn’t Windows), but I’d still be curious to see how Windows Hyper-V VMs scale in terms of performance.
I was able to OC my pair of A4000s by 24% while dropping their draw down to 118W completely stable. Yeah, workstation cards are pricey but you're also getting some of the highest binned chips. It's my home ML rig for a few projects so the design goal was to max VRAM and tensor cores under a low power envelope. Games pretty well when it's not crunching numbers.
Sounds interesting, I use 6 A4000 in my setup, bought them refurbished for $700 each, and run 2 in my Thredripper WS and 2 in each server with Xeon CPUs mainly as render farm for 3D animation in 4 and 8K
@@fordonmekochgalenskaper5665 That sounds interesting. What sort of 3D animation are you doing (eg. game, engineering, advertising...)? I'm interested in how you have them setup to be your render farm because I built this HEDT WS (24c threadripper (TRX40)) with the intent of turning it into a storage/compute server as it ages. I'm presuming the 2x per for all of those systems is similar to my own situation where I only have two full 16x slots. I know that some rendering work is very similar in how it taxes bandwidth so I'd love it if you had any insights about their comparative performance for that sort of workload on 8x vs 16x. I bought mine during the scalper shortage and they were new at about $1300 each, but so were the other GA104's with the same core counts (3070TIs) so it was a direct tradeoff for larger but slightly slower ram and lower power consumption with no additional cost. The single slot design has some pitfalls and I didn't go so far as to water-cool but I did buy some CNC cut copper plates that do a much better job of linking the ram into the heatsink and am quite happy with them. (That might have been part of what let me OC Despite lacking the NV link available on higher models, I wouldn't mind picking up some more if I can find them refurbished at a decent price.
@@fordonmekochgalenskaper5665 I'm guessing that the 2x per is due to limited 16x slots. And I'm assuming that in terms of bandwidth a rendering workload is relatively similar to ML, with data in perhaps being a bit lighter and data out heavier. Personally, the lack of nvlink is a non-issue compared to on card ram space and I'm not sure how it would affect a rendering workload but I'd be really interested on any insights into these cards relative performance in an 8x slot.
"Some may look at this and say that I have an unhealthy obsession with Enterprise graphics cards, this isn't even half of my collection" Has the same energy of "You're not wrong. you've just underestimated me...." 🤣🤣🤣
I have a similar setup with Dual A5000s and one A4000 in a Fractal North with a 7800x3D and Asus ProArt x670e motherboard. These cards are beasts and low power usage.
I want to run this on a very light scale. It's just my wife and me. We just want to run a Mac for our creative work and browsing the internet. While if we need gaming, we hook into a cloud VM and play on that.
I picked up a used A5000 two Christmases back, for 1k, back when the 40 series just came out and presumeably miners were all dumping their old cards. It is a little jank on it's display ports but it's been amazing. I've just recently started playing with partitioning on virtual machines. With my e52689 v4, loaded with 128 gb of ram, I feel like I'm in home lab heaven. What an amazing card. =)
Any chance for a budget oriented follow up video? Id like to have a single virtual gaming vm as part of my proxmox stack. A guide and hardware list might be a good video for the future. Great video though! Its cool to see these types of configurations even if the licensing and cost put them out of homelab reach.
This is great. I feel like the secret to the old budget "CHAD-well" system to get slightly better individual VM performance would likely be to use MULTIPLE P4s instead of fewer more powerful cards... individual SSDs for OS, then perhaps a POOL for slower bulk storage for older smaller games. The Epyc and A5000s are definitely pie in the sky territory. It looks like Crazy fun, but I would be lucky if I saw that kind of GPU hardware in a work setting. But you know that. We will continue to be scrappy on the budget end and give Haswell/Bradwell & Pascal the BEST opportunities we can to optimizing the storage and VM resources. I may also be exploring cheap used gaming PC hardware as a "stack" Ryzen 3600/5600 and i7 8700/9700 gaming PCs with GTX 1650/ RTX 3050s or A2000s might give VMs on the HIGH end a run for their money... they would be expensive potentially, but they could be bought and assembled one good ebay/FB marketplace deal at a time. That or perhaps Erying 12th/13th gen micro ATX DDR4 boards...
The benefit of higher-end hardware is the bigger flexibility. Sure, you might get some more performance per dollar if one GPU doesnt have to juggle half a dozen VMs, but fewer more powerful GPUs allow you to get much higher performance in one instance if you dont need the others. E.g. Im running two VMs on a single RX 6900xt (and will be trying out 3 VMs this weekend), but alot of time its only me using the computer. In that case I still have access to all the compute power the graphics card (and CPU) offer and can enjoy 4k gaming, then scale back the details once the second VM is added.
@@ProfTheorie oh of course. I mostly look at these as a dedicated device and targeting efficiency and a minimum spec. ASSUMING the P4 is enough for the configuration OR the settings are targeted at the specification of the P4... the game can be locked at a specific FPS and setting and the most you'll consume per GPU is 40-60w. You may not get that wonderful maximum... but you also likely won't fall below a specific performance target as long as something catastrophic happens OR if you over allocate. Jeff's earlier findings suggested that the "all P4" scenario had the best potential at maximum capacity especially with the efficiency AND them only taking up a single slot. ALSO because they happen to be possibly the most cost effective at the moment. I feel there are a few more optimizations to be had at the Pascal/Broadwell pricepoint.
I can vouch for a 3600 based server. I'm running a few Minecraft servers, nextcloud, Plex, and a few other services on a 3600 with 16 GB of RAM and it's very performant and cheap. RAM is the only limiting factor but a 32 or 64 GB kit isn't too ridiculous
I have 2 P40s that I was intending primarily to use for LLMs and cloud gaming and I’m having a really hard time justifying keeping them seeing their market value double in the last 2 months lol
@@insu_na yeah. And it sucks because I know their price is gonna absolutely plummet whenever the next enterprise card in line hits the second hand market. So… use and enjoy them now… or double my money and just set up a significantly weaker AI rig. Hmmm.
I appreciate the educational appraoch of your videos... and I genuinely like you as a youtuber. I've learned a lot from your tutorials and have been a long time follower. but I have to say man, this "cloud gaming" thing you've been chasing, its just... not something a lot of people like.... as a gamer , i just dont like to play videogames via remote cloud coinnections... period... anyways, i'll still watch your (cloud gaming) videos, oout of curiosity, and kindof as an example of what NOT to spend my money in. kindof way, u know?? So keep putting them out there and I'll keep watchin :D lolol Cheers!
As much as i love seeing stuff like this, i cant help but scratch my head at the bang/buck, you are setting up a massive epyc server and spending north of 2K on a GPU that can run 2 instances of cyberpunk at lower performance then i would expect out of a 300 USD gpu (might even be able to beat it with a 200-250 USD rx6600/6650xt), while introducing a lot of complexity, upkeep workload and potential for latency. And that isnt getting into the whole licensing thing. This might have been covered in previous videos, but could you perhaps expand on how you use these cloud gaming machines in real life? Im having a hard time seeing a real world scenario which justifies all the work and cost, compared to purpose built consumer grade machines with a side of commercial cloud gaming services for when flexibility is called for.
I think jeff does these for science/fun type of projects on consumer level use. Most of these systems get built from decommissioned parts from work and get a new life instead of being in the junk pile. In Small business/ work environments. for example a local Civil engineering contractors office or interior design, movie studios working on multiple projects and freelancers, draftsman etc are involved on individual smaller tasks during the day which don't justify buying each of them a dedicated GPU for each. Instead you can setup think clients which are easier to carry and replace. but when you need all the individuals work to be integrated into the final project you reconfigure the vm and schedule the task as you get off work
1600$ for 1 A5000 on Ebay (if you are lucky), plus the epyc or threadripper system, rack, case plus the cost of whatever cheap streaming system you are using to game on. 10,000-20,000$ total? If I had the scrilla Id totally do it. But Im kinda crazy too.
You can build a midrange epyc server for 2k, shove all this in a consumer case in a closet for an extra 200 and be all in for ballpark 5k. Still not cheap but I did a similar build sans the GPU for $800, push to 1500 if you want a crazy GPU but afforadable all things considered
@@jorper2526 I made the range to reflect both a used Epyc or new threadripper system (including ram and storage). But 8k does sound a bit closer to right on the low end.
@zacharywilliams3957 3 A5000s = 4800$ and 2k for the epyc is closer to 7k.... plus extras like streaming hard ware.... just don't see anyone getting this done for anywhere near 5k.
@@Yuriel1981 I was working off the 1600 quote from the other poster for an a5000. Could definitely see it push 8k at that cost. I’m not interested in the a5000, I just threw a 3090 in my build since I didn’t need the vgpu function
The video is excellent but you are leaving out so much content that needs answers! - Are GRID drivers overall worth it over passing through a consumer GPU in proxmox ? - Were you able to get max GPU utilization under load (in benchmarks i'm at 100% but in cyberpunk virtually caped at around 60% on 4090) - Is the head dissipation / power utilization better on A5000 vs 4090 ? - What's the proxmox config SO MUCH QUESTIONS !
GRID isn't really comparable to GPU passthrough, you can only pass 1 GPU to 1 VM, GRID chops the GPU up virtually to allow multiple VMs to access the same GPU, basic GPU passthrough is meant for 1 VM at a time, GRID is meant for multiple at a time
I prefer blower style cards as they push all hot air out of the case. I don’t like consumer solutions that make hot move in the case heating everything up, especially if you have more than 1 GPU.
How do I get alerts about what item you're about to review before the video release so I can make purchases before the Craft Computing effect kicks in???
If you are only doing one VM would you do pass-through or stick to vGPU? Does the wddm registry trick work with this on the latest drivers? My P40 works on the 551.61 or 78 driver. Newer drivers no longer work.
@@CraftComputing Nice, i'll be awaiting those results then. It could be interesting to see some test results under Linux too to compare, maybe there's some more performance to squeeze out.
I bought an Epyc Milan 7573x with 2 x 3090 primarily for machine learning but I might try out the game server capabilities as well. This video gave me good inspiration 😀
does RUclips have the ability to unsubscribe people because I was subscribed to this channel a long time ago and suddenly I wasn't getting any content and then you popped up randomly on my feed today. Anyway, I resubscribed.
What do you actually use as thin clients? I'd love to see a round up of the smallest/cheapest thin clients and what matters? Networking, RAM, core, distro etc.
I definitely recommend using VirtIO drivers and all para virtualization options, you can get better storage numbers. It takes some doing to get windows to boot with this controller on the system disk, but it's worth it
You might wanna try switching the SSD, giving a shot to one or two Intel P4608. They should be a lot more capable of handling a demanding parallel load. Or since you run Linux, ask ScaleFlux if you can try their CSD. I did benchmarks with the prior gen of those and, definitely, they didn't care at all if you put them under higher load. And the new ones are even more powerful.
I with the consumer graphics card would already not have any display output on themselves like those compute cards and either route the output to the board outputs. That pci bracket is cumbersome for SFF systems and should be standardized to be removable/replaceable.
as weird as it sounds, you may be better off with bare metal Linux for older games, unironically Wine/Proton tends to run older games smoother then a windows VM does mainly due to XP just having god awful hardware support even inside a VM, most VM offerings don't optimize for OS's as old as XP so XP can easily freak out when presented with VM "hardware"
I really want to see what a 7950x will do with these gpus ! the cpu clock are so high it will really help the bottleneck and even a 7800x3d and give each vm 8vcore to see how that works !
In addn to hw cost for the A5000, would one also need a separate Grid license payment(s) ? Thank you. I've watched most of these cloud gaming and VDI videos but cannot remember.
If you are homelab user and don't earn money from it you can have fake activation server which giving you licenses for free no matter you are on proxmox or vmware and not matter what your guest OS is.
I would like to start by saying a big THANK YOU! I am new to the home-lab world, and your channel seems to have all the answers to the queries I have. If possible, I would love to see how a budget P1000 Graphics card performs in a single gaming VM and a "Budget" ProxMox build with an EPYC 7002 series 64 core processor which has enough PCIe lanes to accommodate multiple P1000's. Also, I would really appreciate if you could put light on Windows Licensing requirements for multiple VM's. Thank you yet again for the awesome content you've been putting out!
Question: does the cloud gaming setup only work with Nvidia gpus (I can only remembering seeing Nvidia cards in this series) or does it support AMD gpus as well?
I just bought the A5000 on a criminally killer deal (less than $400). I'm sure it will be nice when I use it in Agisoft Metashape and Da Vinci Resolve, but given a similar cost should I swap it for a Titan RTX next year for much better DLSS performance and slightly better gaming & ray tracing performance?
With so much graphics cards can you try steam headless in docker? In theory seems like a great way to make several gaming vms without making virtual machines
I’ve been toying with this as well but I’m sticking to consumer GPUs and having a single GPU per VM. I’d rather have this option but the cost just doesn’t make sense for me.
Have you tried using any of the intel ARC cards with your cloud gaming setup? Im curious if there is anything compatible out there as it seems they may be a viable option.
I run unraid, and would love to share a good GPU between simultaneous windows VM's that i remote desktop into but it isnt a feature supported by unraid unfortunately. Maybe some day? Setting unraid up within proxmox is a step to far for me. I dont have the technical skills.
I've already tried splitting 3090s using Microsoft Hyper V, as a result of your prior video(s). (Thank you, BTW!) I couldn't get the software (Sunshine/Moonlight) to play nicely with my system (5950X with 128 GB of RAM), and Halo Infinite ended up being mostly a go-no. (Also couldn't maintain a stable connection over hardwire GbE LAN, using Parsec.) That, plus Easy Anti-Cheat hating running games that use/need it, via a VM = no-go for a lot of the games where I could've really used it. For these reasons, the gaming systems are moving into 3U rackmounts (in the future), instead or mini ITX/SFF instead. (Tiny human #1 right now, uses a NUC with a Razer Core X eGPU as a gaming system, so moving all of that into something like the Fractal Terra would actually shrink the volume down by ~3 L, and it would probably be more performant as the GPU won't be limited by the Thunderbolt 3 link, and would have full, native, PCIe 4.0 x16 speeds (depending on which motherboard and CPU I choose, along with the CPU HSF).) So it won't be difficult to deploy a bunch of these systems throughout the house (or if I want to have a kid's play room with their toys and the gaming computers, then I can deploy a rack or another rack, and then just run it from there). I love the idea of remote and/or virtualised gaming. But things like EAC doesn't make that always necessarily easy nor possible.
Please could you test running 3 instances of MSFS 2020 in 4K? As this is what I need to do for my sim. I'd be so grateful if you could test. Thank you.
The m.2 ssd tests, would you not hit the zfs arc on proxmox for using the same generate file again giving these results? (Don't know how the vmdisk abstraction would work but my proxmox zfs pool still has an arc cache that is being hit). Can't you turn off arc on the zfs pool since it would still be faster than ram?
@CraftComputing For my Windows VMs under Proxmox I wasn’t able to get very good speeds until I switched to the Virtio storage drivers. Doing this switched the VM from getting similar to what you had around 3000 MB/s up to the max of my gen four drive of 7000 MB/s! Not sure how this works for a boot drive. I imagine this would require locating a driver during install. I did this on a secondary drive to benchmark drive speeds from a VM.
There is a new open-source Kernel Module from Nvidia but it only supports Turing, Ampere or newer. I am not sure if Proxmox already supports it, but it should be a lot easier to share let’s say an RTX 2080 Ti in the near future…
@@CraftComputing I have a topic suggestion / video idea: most people (myself included until recently) have no clear idea about how Virtual GPU Licensing works, why you may prefer GPU-pass through to have good performance and avoid expensive licenses, why it may be better / preferable to have a regular Linux server installation on the host and use KVM Virtualization instead of bare-metal like Proxmox to avoid dealing with vGPU licensing (for everything except gaming, including most workstation workloads); then do a 2nd video to talk about specific video card models for doing multiple GPU passthrough on a single server (i.e. when it is better to have 3x RTX 4000 instead of a Tesla GPU installed on a single server); then do a 3rd video to talk about the future of SR-IOV. The idea is to make 10min. videos discussing the options, not a How-to video which you already have on your channel… Thanks for making great content!
I have a question, is it possible to run the RTX8000 vGPU license from NVIDIA in a one computer? I want to use it for 3D work and I don’t want to go through running two pcs (Host server and virtual workstation). I want to access it directly from my pc which has RTX 3070! Any advice please?
I just bought a bargain basement Tesla M40 and SMOKED IT at the first power on. The company I bought it from didn't mention or include the special power cable and a standard pcie 8 pin connector fits right in. Serves me right for not doing my research. Next time, definitely...
It requires 8-Pin EPS connector, the same as your CPU motherboard connector. PCIe power is keyed differently and should not have fit without encouragement.
@@CraftComputing Too bad I figured that out after the blue smoke. I have an older proprietary server that came with no such connector from the power supply so I bought a 6 pin to 8 pin adapter. The pcie connector from the el-cheapo adapter plugged right in with no force required. But I will say I take full responsibility for not researching better. Good thing the card cost less than $100. I hope I am not the only one to let this particular genie out of the bottle.
I didn’t like the ambiguous “too expensive” comment. How much is grid licensing? Quick search made it sound like $100/user but I might be missing something?
Im really excited you mentioned using this for workstation. This is exactly the type of system I have been dreaming of. Multiple instances of VMs to avoid dual boot. Running CAD for engineering business. I never want to worry about software updates on those VMs and launch multiple instances for different projects. Thanks
Doesn't AMD has something alike coming? I remember i saw something about it where AMD said they would release a driver that would allow for vGPU in a free and open-source maner
They've touted their MxGPU products as 'license free', but they don't actually provide a download to them without support contracts..... So...... 🤷 Intel Flex is also licensed free, but good luck finding a card.
im a simple nerd. i see a cloud gaming server - i press like.
AYE
Me too. It's pretty much the dream I've had for 30+ years.
We've had the hardware to do what I want for a while, we just needed the software not to be hidden behind massive paywalls.
True that! 🦾
I'm a simple commenter. I see someone liking a video and I press like on the comment.
00:04 - yes.
I do too, but can't afford them so I come here for the dopamine hit. 😁
Just wanted to give my heartfelt thanks to you Jeff. A while back I discovered your channel and have binge-watched most of your videos regarding cloud gaming, virtualisation and server hardware since then. Your videos (especially the Hyper-V paravirtualisation) allowed me to get my sort-of gaming-server (Im still sitting on it bare-metal but also hosting VMs and streaming all over the place) using Hyper-V GPU Paravirtualisation going, something I wanted to do for over a decade. 2 simultaneous games? No problem. Streaming an emulator running in a vm to a computer 300km away? Works just fine. Incidentially this has also revitalised some old laptops that can now be given some computing power.
Ill be testing out 3x Baldurs Gate on one 6900XT + 7900X3D this weekend, lets hope it works. If it does 2 people will be saving quite some money not having to buy a computer.
Yes, please cover virtualization on consumer Ampere, that would be sick! The RTX 3090 is typically hundreds of dollars cheaper on the used market than its A-series VRAM equivalents, but with even more cores!
You can't split the consumer Ampere cards, you can only use full gpu for one vm
@@pachete. That’s not strictly true; LTT posted a video of them splitting a 3080ti into two VMs two years ago using Windows Hyper-V. I know that isn’t as appealing as just running the VM’s through a more lightweight OS (or frankly, any OS that isn’t Windows), but I’d still be curious to see how Windows Hyper-V VMs scale in terms of performance.
I was able to OC my pair of A4000s by 24% while dropping their draw down to 118W completely stable. Yeah, workstation cards are pricey but you're also getting some of the highest binned chips. It's my home ML rig for a few projects so the design goal was to max VRAM and tensor cores under a low power envelope.
Games pretty well when it's not crunching numbers.
Sounds interesting, I use 6 A4000 in my setup, bought them refurbished for $700 each, and run 2 in my Thredripper WS and 2 in each server with Xeon CPUs mainly as render farm for 3D animation in 4 and 8K
@@fordonmekochgalenskaper5665 That sounds interesting. What sort of 3D animation are you doing (eg. game, engineering, advertising...)? I'm interested in how you have them setup to be your render farm because I built this HEDT WS (24c threadripper (TRX40)) with the intent of turning it into a storage/compute server as it ages. I'm presuming the 2x per for all of those systems is similar to my own situation where I only have two full 16x slots. I know that some rendering work is very similar in how it taxes bandwidth so I'd love it if you had any insights about their comparative performance for that sort of workload on 8x vs 16x.
I bought mine during the scalper shortage and they were new at about $1300 each, but so were the other GA104's with the same core counts (3070TIs) so it was a direct tradeoff for larger but slightly slower ram and lower power consumption with no additional cost. The single slot design has some pitfalls and I didn't go so far as to water-cool but I did buy some CNC cut copper plates that do a much better job of linking the ram into the heatsink and am quite happy with them. (That might have been part of what let me OC Despite lacking the NV link available on higher models, I wouldn't mind picking up some more if I can find them refurbished at a decent price.
@@fordonmekochgalenskaper5665 I'm guessing that the 2x per is due to limited 16x slots. And I'm assuming that in terms of bandwidth a rendering workload is relatively similar to ML, with data in perhaps being a bit lighter and data out heavier. Personally, the lack of nvlink is a non-issue compared to on card ram space and I'm not sure how it would affect a rendering workload but I'd be really interested on any insights into these cards relative performance in an 8x slot.
Quadro/Tesla cards used to come with overclocking disabled. Wierd they are now allowing OC.
@@3DJLab it is quite limited but not completely locked out.
"Some may look at this and say that I have an unhealthy obsession with Enterprise graphics cards, this isn't even half of my collection"
Has the same energy of "You're not wrong. you've just underestimated me...." 🤣🤣🤣
Cheers! Great video as always.
I have a similar setup with Dual A5000s and one A4000 in a Fractal North with a 7800x3D and Asus ProArt x670e motherboard. These cards are beasts and low power usage.
I want to run this on a very light scale. It's just my wife and me. We just want to run a Mac for our creative work and browsing the internet. While if we need gaming, we hook into a cloud VM and play on that.
I picked up a used A5000 two Christmases back, for 1k, back when the 40 series just came out and presumeably miners were all dumping their old cards. It is a little jank on it's display ports but it's been amazing. I've just recently started playing with partitioning on virtual machines. With my e52689 v4, loaded with 128 gb of ram, I feel like I'm in home lab heaven. What an amazing card. =)
I started a home lab because of your channel. I love what I can do. Thank you.
Any chance for a budget oriented follow up video? Id like to have a single virtual gaming vm as part of my proxmox stack. A guide and hardware list might be a good video for the future.
Great video though! Its cool to see these types of configurations even if the licensing and cost put them out of homelab reach.
It's called steam remote play 🤣
This is great. I feel like the secret to the old budget "CHAD-well" system to get slightly better individual VM performance would likely be to use MULTIPLE P4s instead of fewer more powerful cards... individual SSDs for OS, then perhaps a POOL for slower bulk storage for older smaller games. The Epyc and A5000s are definitely pie in the sky territory. It looks like Crazy fun, but I would be lucky if I saw that kind of GPU hardware in a work setting. But you know that. We will continue to be scrappy on the budget end and give Haswell/Bradwell & Pascal the BEST opportunities we can to optimizing the storage and VM resources.
I may also be exploring cheap used gaming PC hardware as a "stack"
Ryzen 3600/5600 and i7 8700/9700 gaming PCs with GTX 1650/ RTX 3050s or A2000s might give VMs on the HIGH end a run for their money... they would be expensive potentially, but they could be bought and assembled one good ebay/FB marketplace deal at a time. That or perhaps Erying 12th/13th gen micro ATX DDR4 boards...
The benefit of higher-end hardware is the bigger flexibility. Sure, you might get some more performance per dollar if one GPU doesnt have to juggle half a dozen VMs, but fewer more powerful GPUs allow you to get much higher performance in one instance if you dont need the others.
E.g. Im running two VMs on a single RX 6900xt (and will be trying out 3 VMs this weekend), but alot of time its only me using the computer. In that case I still have access to all the compute power the graphics card (and CPU) offer and can enjoy 4k gaming, then scale back the details once the second VM is added.
@@ProfTheorie oh of course. I mostly look at these as a dedicated device and targeting efficiency and a minimum spec. ASSUMING the P4 is enough for the configuration OR the settings are targeted at the specification of the P4... the game can be locked at a specific FPS and setting and the most you'll consume per GPU is 40-60w. You may not get that wonderful maximum... but you also likely won't fall below a specific performance target as long as something catastrophic happens OR if you over allocate. Jeff's earlier findings suggested that the "all P4" scenario had the best potential at maximum capacity especially with the efficiency AND them only taking up a single slot. ALSO because they happen to be possibly the most cost effective at the moment. I feel there are a few more optimizations to be had at the Pascal/Broadwell pricepoint.
I can vouch for a 3600 based server. I'm running a few Minecraft servers, nextcloud, Plex, and a few other services on a 3600 with 16 GB of RAM and it's very performant and cheap. RAM is the only limiting factor but a 32 or 64 GB kit isn't too ridiculous
I'm wondering if you have had experience and benchmarks with VirtIO (particularly Venus for vulkan passthrough), linux guests etc.
this is bonkers. I have been wanting to make a cloud gaming server myself
I have 2 P40s that I was intending primarily to use for LLMs and cloud gaming and I’m having a really hard time justifying keeping them seeing their market value double in the last 2 months lol
Ha, same. Incredible how much they increased in price, and for no apparent reason
@@insu_na yeah. And it sucks because I know their price is gonna absolutely plummet whenever the next enterprise card in line hits the second hand market.
So… use and enjoy them now… or double my money and just set up a significantly weaker AI rig. Hmmm.
I appreciate the educational appraoch of your videos... and I genuinely like you as a youtuber. I've learned a lot from your tutorials and have been a long time follower.
but I have to say man, this "cloud gaming" thing you've been chasing, its just... not something a lot of people like.... as a gamer , i just dont like to play videogames via remote cloud coinnections... period...
anyways, i'll still watch your (cloud gaming) videos, oout of curiosity, and kindof as an example of what NOT to spend my money in. kindof way, u know?? So keep putting them out there and I'll keep watchin :D lolol
Cheers!
As much as i love seeing stuff like this, i cant help but scratch my head at the bang/buck, you are setting up a massive epyc server and spending north of 2K on a GPU that can run 2 instances of cyberpunk at lower performance then i would expect out of a 300 USD gpu (might even be able to beat it with a 200-250 USD rx6600/6650xt), while introducing a lot of complexity, upkeep workload and potential for latency. And that isnt getting into the whole licensing thing.
This might have been covered in previous videos, but could you perhaps expand on how you use these cloud gaming machines in real life? Im having a hard time seeing a real world scenario which justifies all the work and cost, compared to purpose built consumer grade machines with a side of commercial cloud gaming services for when flexibility is called for.
I think jeff does these for science/fun type of projects on consumer level use. Most of these systems get built from decommissioned parts from work and get a new life instead of being in the junk pile.
In Small business/ work environments. for example a local Civil engineering contractors office or interior design, movie studios working on multiple projects and freelancers, draftsman etc are involved on individual smaller tasks during the day which don't justify buying each of them a dedicated GPU for each.
Instead you can setup think clients which are easier to carry and replace.
but when you need all the individuals work to be integrated into the final project you reconfigure the vm and schedule the task as you get off work
1600$ for 1 A5000 on Ebay (if you are lucky), plus the epyc or threadripper system, rack, case plus the cost of whatever cheap streaming system you are using to game on. 10,000-20,000$ total? If I had the scrilla Id totally do it. But Im kinda crazy too.
You can build a midrange epyc server for 2k, shove all this in a consumer case in a closet for an extra 200 and be all in for ballpark 5k. Still not cheap but I did a similar build sans the GPU for $800, push to 1500 if you want a crazy GPU but afforadable all things considered
I think closer to $8K.. Though at that price, if you can drop $8K on a system, you can drop $10K on a system lol.
@@jorper2526 I made the range to reflect both a used Epyc or new threadripper system (including ram and storage). But 8k does sound a bit closer to right on the low end.
@zacharywilliams3957 3 A5000s = 4800$ and 2k for the epyc is closer to 7k.... plus extras like streaming hard ware.... just don't see anyone getting this done for anywhere near 5k.
@@Yuriel1981 I was working off the 1600 quote from the other poster for an a5000. Could definitely see it push 8k at that cost. I’m not interested in the a5000, I just threw a 3090 in my build since I didn’t need the vgpu function
The video is excellent but you are leaving out so much content that needs answers!
- Are GRID drivers overall worth it over passing through a consumer GPU in proxmox ?
- Were you able to get max GPU utilization under load (in benchmarks i'm at 100% but in cyberpunk virtually caped at around 60% on 4090)
- Is the head dissipation / power utilization better on A5000 vs 4090 ?
- What's the proxmox config
SO MUCH QUESTIONS !
GRID isn't really comparable to GPU passthrough, you can only pass 1 GPU to 1 VM, GRID chops the GPU up virtually to allow multiple VMs to access the same GPU, basic GPU passthrough is meant for 1 VM at a time, GRID is meant for multiple at a time
I prefer blower style cards as they push all hot air out of the case. I don’t like consumer solutions that make hot move in the case heating everything up, especially if you have more than 1 GPU.
How do I get alerts about what item you're about to review before the video release so I can make purchases before the Craft Computing effect kicks in???
Join the Patreon/Discord, you'll know what projects I'm working on so you can buy early ;-)
If you are only doing one VM would you do pass-through or stick to vGPU? Does the wddm registry trick work with this on the latest drivers? My P40 works on the 551.61 or 78 driver. Newer drivers no longer work.
How many Crysis instances you can run? The question is still not answered -.-
I actually tried to make that happen in this video, but Crysis refused to load in the latest W11. Don't worry, I will be exploring that :-)
@@CraftComputing Nice, i'll be awaiting those results then. It could be interesting to see some test results under Linux too to compare, maybe there's some more performance to squeeze out.
Infinite
😂
I bought an Epyc Milan 7573x with 2 x 3090 primarily for machine learning but I might try out the game server capabilities as well. This video gave me good inspiration 😀
Are you happy with this setup? I'm considering it myself for ML/DL as well.
@@yakovdanthere is a Puget systems HPC article on threadripper using 4 3090s tuned to sip 180w to 122w for best results. Dr. Kinghorn is the best
you could use that article for reference if Op is busy @@yakovdan
The dedication to your cloud gaming project is commendable.
Probably would've driven me crazy before the first year ended lol
I have a VDI cluster at work with 8 Grid M10 GPUs that i can't wait to decom. 4x Maxwell GPUs with 32GB RAM total per card.
I'm trying to calculate this in my head...that's Alot of GPU and CPU and storage. Interesting in the ever changing build.
does RUclips have the ability to unsubscribe people because I was subscribed to this channel a long time ago and suddenly I wasn't getting any content and then you popped up randomly on my feed today. Anyway, I resubscribed.
What do you actually use as thin clients? I'd love to see a round up of the smallest/cheapest thin clients and what matters? Networking, RAM, core, distro etc.
on the go he has a GPD win max I believe with 6800U.
you could use a steam deck
3:43 watch the glass for that jump cut swig, ☺
I definitely recommend using VirtIO drivers and all para virtualization options, you can get better storage numbers. It takes some doing to get windows to boot with this controller on the system disk, but it's worth it
Yeah, I'm going to be looking at VirtIO shortly for the storage drivers.
i would love to see what firepro and arc flex cards can do in similar testing. maybe even radeon and the a770?
You might wanna try switching the SSD, giving a shot to one or two Intel P4608. They should be a lot more capable of handling a demanding parallel load.
Or since you run Linux, ask ScaleFlux if you can try their CSD.
I did benchmarks with the prior gen of those and, definitely, they didn't care at all if you put them under higher load. And the new ones are even more powerful.
I with the consumer graphics card would already not have any display output on themselves like those compute cards and either route the output to the board outputs. That pci bracket is cumbersome for SFF systems and should be standardized to be removable/replaceable.
I love your gaming virtualization videos. All I want is an XP VM to play my old games. 😥
I’ve tried to do this, but ran into issues with UEFI being needed for GPU passthrough, and not able to install XP
as weird as it sounds, you may be better off with bare metal Linux for older games, unironically Wine/Proton tends to run older games smoother then a windows VM does mainly due to XP just having god awful hardware support even inside a VM, most VM offerings don't optimize for OS's as old as XP so XP can easily freak out when presented with VM "hardware"
I really want to see what a 7950x will do with these gpus ! the cpu clock are so high it will really help the bottleneck
and even a 7800x3d and give each vm 8vcore to see how that works !
LOL ... Statement: "Some may say I have an obsession" Response: "Yes"
In addn to hw cost for the A5000, would one also need a separate Grid license payment(s) ? Thank you. I've watched most of these cloud gaming and VDI videos but cannot remember.
Yes, you need Grid licensing on top of the hardware cost. I'm using a trial license here.
@@CraftComputing Thank you. Have an outstanding weekend.
If you are homelab user and don't earn money from it you can have fake activation server which giving you licenses for free no matter you are on proxmox or vmware and not matter what your guest OS is.
@@jakubmad3957 thanks
Id be interested to see the a5000 vs an rtx titan. Ive been looking at that as a 'cheaper' alternative it an a5000
I would like to start by saying a big THANK YOU! I am new to the home-lab world, and your channel seems to have all the answers to the queries I have. If possible, I would love to see how a budget P1000 Graphics card performs in a single gaming VM and a "Budget" ProxMox build with an EPYC 7002 series 64 core processor which has enough PCIe lanes to accommodate multiple P1000's. Also, I would really appreciate if you could put light on Windows Licensing requirements for multiple VM's. Thank you yet again for the awesome content you've been putting out!
Actively building a *very* similar system to this with an Epyc 9334
Question: does the cloud gaming setup only work with Nvidia gpus (I can only remembering seeing Nvidia cards in this series) or does it support AMD gpus as well?
I just bought the A5000 on a criminally killer deal (less than $400).
I'm sure it will be nice when I use it in Agisoft Metashape and Da Vinci Resolve, but given a similar cost should I swap it for a Titan RTX next year for much better DLSS performance and slightly better gaming & ray tracing performance?
With so much graphics cards can you try steam headless in docker? In theory seems like a great way to make several gaming vms without making virtual machines
I’ve been toying with this as well but I’m sticking to consumer GPUs and having a single GPU per VM. I’d rather have this option but the cost just doesn’t make sense for me.
i trhink the huge 1% low improvement is caused by the ampere cards using sr-iov
I would be very interested to see how a genoa Epyc with 3d vchace performs in such task
Can you pass through each disk to a VM? … Do nvme drives pass through like a PCIe device?
You absolutely can.
@@CraftComputing but can youuuu do it and let us know if there is a performance improvement? 😁
Damn, so cool! Meanwhile me struggling to run windows at 60fps haha.
How do you connect to your remote machines? Do you use Windows remote desktop or something else?
Have you tried using any of the intel ARC cards with your cloud gaming setup? Im curious if there is anything compatible out there as it seems they may be a viable option.
I have not yet. I've been waiting to get a hold of a Flex 170 card that supports SR-IOV for testing.
I run unraid, and would love to share a good GPU between simultaneous windows VM's that i remote desktop into but it isnt a feature supported by unraid unfortunately. Maybe some day?
Setting unraid up within proxmox is a step to far for me. I dont have the technical skills.
I have 2 RTX 3090s paravirtualizing to 2 VMs each. Working on a write-up RN. 13700K + 128GB DDR4.
Where can I find this when it's done?
if you love what your doing nothing is unhealthy. i love watching your videos doing stuff i can only dream of doing. you are living the dream dude
curious what its ai benchmarks are
one of these days you will introduce yourself as someone OTHER than Jeff in the beginning of a video and we will all be like "so NOT always???"
I've already tried splitting 3090s using Microsoft Hyper V, as a result of your prior video(s). (Thank you, BTW!)
I couldn't get the software (Sunshine/Moonlight) to play nicely with my system (5950X with 128 GB of RAM), and Halo Infinite ended up being mostly a go-no.
(Also couldn't maintain a stable connection over hardwire GbE LAN, using Parsec.)
That, plus Easy Anti-Cheat hating running games that use/need it, via a VM = no-go for a lot of the games where I could've really used it.
For these reasons, the gaming systems are moving into 3U rackmounts (in the future), instead or mini ITX/SFF instead. (Tiny human #1 right now, uses a NUC with a Razer Core X eGPU as a gaming system, so moving all of that into something like the Fractal Terra would actually shrink the volume down by ~3 L, and it would probably be more performant as the GPU won't be limited by the Thunderbolt 3 link, and would have full, native, PCIe 4.0 x16 speeds (depending on which motherboard and CPU I choose, along with the CPU HSF).)
So it won't be difficult to deploy a bunch of these systems throughout the house (or if I want to have a kid's play room with their toys and the gaming computers, then I can deploy a rack or another rack, and then just run it from there).
I love the idea of remote and/or virtualised gaming.
But things like EAC doesn't make that always necessarily easy nor possible.
Jeff, we need a cloud gaming server LAN party video!
Please could you test running 3 instances of MSFS 2020 in 4K? As this is what I need to do for my sim. I'd be so grateful if you could test. Thank you.
you should do some videos about Intel Arc GPUs SR-IOV.
I'd love to. Intel won't send them out.
I have a 3090 on a B550 and an A4000 and a 6148 on a c621-64L SAGE on different computers. I am having intrusive thoughts.
Aree you getting PICe 5.0 speeds on a Gigabyte gen 4 x4 riser?
That card is the new Gen5 riser. Yes, I tested single disk speeds at 12GB/s.
The m.2 ssd tests, would you not hit the zfs arc on proxmox for using the same generate file again giving these results? (Don't know how the vmdisk abstraction would work but my proxmox zfs pool still has an arc cache that is being hit). Can't you turn off arc on the zfs pool since it would still be faster than ram?
Would be VERY interested in the Hyper-V tests!!!
If you're going to test HyperV, wait until Server 2025 launches, it brings a ton of updates to device assignment. Should be fun!
I've absolutely been keeping an eye on that. The GUI for configuring GPU-P has entered the insider preview 🤘
@@CraftComputing awesome!
I think the real question is how many clients can you host on a GB200 NVL72?
My laptop has an RTX 5000 mobile gpu with 16 gb vram... If you had one of those, would you try to turn that into a cloud gaming server too?
Oooh A5000, fancy.
Are they genuinely approaching affordable now? I must have not been paying attention.
They're still around $1000 used here. I wouldn't call them affordable, but they're definitely fast.
is nvidia grid also great for opengl webasm workloads?
@CraftComputing For my Windows VMs under Proxmox I wasn’t able to get very good speeds until I switched to the Virtio storage drivers. Doing this switched the VM from getting similar to what you had around 3000 MB/s up to the max of my gen four drive of 7000 MB/s! Not sure how this works for a boot drive. I imagine this would require locating a driver during install. I did this on a secondary drive to benchmark drive speeds from a VM.
Yes, I'd need to inject the storage drivers during Windows Install, but absolutely should be possible. It's on my list to check out soon.
can you map one of the displayport outputs to a vgpu instance?
There is a new open-source Kernel Module from Nvidia but it only supports Turing, Ampere or newer. I am not sure if Proxmox already supports it, but it should be a lot easier to share let’s say an RTX 2080 Ti in the near future…
I did see that announcement, but haven't tried it out yet. It does still require a license to run unfortunately.
@@CraftComputing I have a topic suggestion / video idea: most people (myself included until recently) have no clear idea about how Virtual GPU Licensing works, why you may prefer GPU-pass through to have good performance and avoid expensive licenses, why it may be better / preferable to have a regular Linux server installation on the host and use KVM Virtualization instead of bare-metal like Proxmox to avoid dealing with vGPU licensing (for everything except gaming, including most workstation workloads); then do a 2nd video to talk about specific video card models for doing multiple GPU passthrough on a single server (i.e. when it is better to have 3x RTX 4000 instead of a Tesla GPU installed on a single server); then do a 3rd video to talk about the future of SR-IOV. The idea is to make 10min. videos discussing the options, not a How-to video which you already have on your channel… Thanks for making great content!
VDI need Raid10 for the lower overhead.
Have you ran I to DRM issue with it running in a vm?
Gotta admire a sponsor that's prepared to fund a video describing how to replace their product 😀
For just $19,000 in server gear, you can save $7.40 per month!
Is there any way to do this with an A4000?
I have a question, is it possible to run the RTX8000 vGPU license from NVIDIA in a one computer? I want to use it for 3D work and I don’t want to go through running two pcs (Host server and virtual workstation). I want to access it directly from my pc which has RTX 3070! Any advice please?
p40 up to $300+ on ebay since your last video
...im sorry 😞
P40 is not that great for $200+, I got one for ~$100 and it is on similar levels as 3060. Main benefit is the 24GB VRAM
Does Steam link work from VM?
how does cloud gaming work on servers but if you make one for home use having 4 people run off one pc the anticheat want let you play it?
0:00
Nah, you have a problem of not ENOUGH! 😂
this is the way
Unhealthy? I don’t think so. Expensive maybe but unhealthy no
Prices are around 2400€ in France on ebay :(
What kind of speeds on the nvme if you dedicate an nvme to each vm?
I just bought a bargain basement Tesla M40 and SMOKED IT at the first power on. The company I bought it from didn't mention or include the special power cable and a standard pcie 8 pin connector fits right in. Serves me right for not doing my research. Next time, definitely...
It requires 8-Pin EPS connector, the same as your CPU motherboard connector. PCIe power is keyed differently and should not have fit without encouragement.
@@CraftComputing Too bad I figured
that out after the blue smoke. I have an older proprietary server that came with no such connector from the power supply so I bought a 6 pin to 8 pin adapter. The pcie connector from the el-cheapo adapter plugged right in with no force required. But I will say I take full responsibility for not researching better. Good thing the card cost less than $100. I hope I am not the only one to let this particular genie out of the bottle.
Ouch, 🍻
how do i do this at home please?
As someone that also have expensive ideas but no money, this channel it's a godsend lol
"...this isn't even half of my collection"
My genuine response that intro: "(chuckle) I love you dude"
Best intro in the business ❤️
instead of z1 run mirrored vdevs..try that.
can you run newer games with "anti cheat" technologies on a prox mox server?
You'll need to check with the anticheat software. They should state whether virtualization is allowed or not. It's going to be a case-by-case basis.
You need an I-Rack for all those GPU's.
I didn’t like the ambiguous “too expensive” comment. How much is grid licensing? Quick search made it sound like $100/user but I might be missing something?
Probably vPC license which afaik has 2gb vram limit, vDWS has 24gb limit. And there is perpetual and subscription licenses.
It's closer to $600/user per year.
Im really excited you mentioned using this for workstation. This is exactly the type of system I have been dreaming of. Multiple instances of VMs to avoid dual boot. Running CAD for engineering business. I never want to worry about software updates on those VMs and launch multiple instances for different projects. Thanks
Nice
Does anyone know how Jeff is connecting to the virtual machines from his thin clients?
Sunshine Game Streaming on the VMs
Moonlight on the client.
@@CraftComputing thank you! I'll take a look at these.
Doesn't AMD has something alike coming? I remember i saw something about it where AMD said they would release a driver that would allow for vGPU in a free and open-source maner
They've touted their MxGPU products as 'license free', but they don't actually provide a download to them without support contracts..... So...... 🤷
Intel Flex is also licensed free, but good luck finding a card.
Always want more of this!
can someone tally how many videos he has with the flat cap vs without the hat
Hell yeah!!! More need to see!