Dude, so much better and clear sense than other Proxmox videos\guides, no forking aroung like any other guide with no clear narration and usefull knowledge.
Thanks for the videos. I know some other guys are "more popular" to watch for proxmox but there's nobody that does the depth you do and we really appreciate it.
Thanks for the video! but for your next videos could you please use diagram software to illustrate complex concepts, it definitely helps the community as all other youtubers use it and it's a must in the networking world :)
Excellent video. As someone that started my own homelab and IT journey with 486s in the late 90s and pushed myself ever since, I appreciate you taking the initiative to share this with the community! Gotta keep this stuff in the hands of everyone to learn and build upon it, the "cloud" mentality these days will only destroy what so many have built. Your Ceph on Proxmox video was far more in-depth than one I watched from a large professional outfit (not mentioning any names because they do have a lot of good videos).
Glad you enjoyed it! I definitely like keeping things locally hosted, even if it's just for 'fun'. Hope you enjoy some of the upcoming projects I have!
Superb. I'm going to have to watch this on slow about 5 times just to get my head around this whole area of Proxmox I knew so little about 🤯 As the song says : "The more I find out, the less I know" 😁 Thankyou.
The imagery of a Linux bridge being "a network switch" and plugging the network interface into it virtually was really helpful! Thanks for that description.
I’ve watched, what feels like at least, every video on RUclips trying to understand how these concepts work in Proxmox, and this was by far the best and explained every question and issue I had in a single video. Thank you so so so much!
4:06 FINALLY someone on fricking RUclips explains that! I was on several "network videos" about Proxmox before and they didn't explained me that concept of vSwitches like you did THANKS MAN ❤
I appreciate your level of thoroughness on the topic at hand. I also appreciate how you will make mention to other related things to bring about awareness without veering off down a rabbit hole or just omitting them altogether. It's a fine balance and I think you've found it!
Hi There, I have 2 dell r720 running pve 8.2.4, connected by 2 1gig links to a cisco 3750 switch. I can get one pve to connect to the switch using lacp for the bond0. the other pve won't connect. same configuration for both bond0s. If you want more info let me know thx Hayward
You did a great way of explaining the networking in the most concise way in proxmox, navgiating through the what ifs as well. Ive been kicking myself for days trying to configure vlanning woth ceph and the mgmt ip and so forth. Homelab security is a must.
@apalrdsadventures How do you diagnose issues when a seemingly simple change breaks this? I have trunked VLANs on 1G (pfSense) and a (GS748Tv5) smart switch. I also have a working bonded LACP link between the switch and a NAS, so I am pretty confident the switch is ok. On proxmox, as soon as I convert the trunk from a NIC port to a bonded NIC (even one), nothing goes through. :( I DID notice that you had to tear it down and build it again to get it to work. I've done that but no joy. Ideas? THANKS!
Very good video. Thanks! If you using bonding bonds check that you are not using VLANs on bond0. I have bond0 (LAGG 10G) and bond1 (backup to 1G). And bond1 not working until I remove all VLANs on bond0.
Thank you! But I don't understand how to make a access from Inet to my virtual machine, and make my VM isolated from all other my network. Yes, maybe VLANs.... But, o my Gos, am I have to block traffic by all to all (subnets) for every new one subnet (group of virtual machines)?
Great video. Finally sat down to re-do my 10G networking and figured it was time to setup active-backup and vlan awareness. When I did the initial setup, my VMs were fine on another 10G bridge I created but my NFS and iSCSI shares were capped at the 1G speeds - Not anymore! Covered exactly what I needed.
Yo dawg. Nice video 😉 Proxmox GUI has come a long way since v4. Was great to see you showing off the possibilities and no config editing. I have the task of creating a bond with 10GbE and 1GbE backup, so your video was perfect to help me dry run and visualise how to achieve this without config editing 👍 no doubt this will save me a bunch of time. You've done a great job of making more advanced network topics accessible to a lot of folks. Bravo.
The VLAN are working wehn I don't have VLAN aware checked. As soon as I check it, it quits working. Also if I migrate the VM to the same node in the cluster as teh pfsense, it quits working. I actually have to shut it down, reboot the node that the pfsense vm is running in for it to start working properly again. I've spent many hours today trying to figure out what's wrong lol. I have 3 vlans and native and pfsense is routing it all properly. But as soon as I migrate a vlan tagged VM to the same node as pfsense or set VLAN aware = yes then the doesn't route the traffic.
this is a great tutorial. I have often struggled with this fumbling till it works. The only thing that would have been more helpful is if you went in a little more on the trunk for the vm... I didn't quite follow that.
That was super cool, now you've got me shopping for managed switches so I can get goofy my home network. I've got 4-port cards in all of my infrastructure boxes already...
This was quite helpful in configuring proxmox for a pfSense VM that has 3 vlans on a trunk. I missed it in your video, but there was a hint to what I needed to complete the configuration. My host PC has 4x 2.5gb ports, and I wanted to have pfSense serve both the trunk vlans and the local ports with their respective DHCP pools. The bridge was the answer! I was able to bridge the vlan to the local port, with the bridge having the IP address and DHCP server, and the vlan and local port having no IPs.
My Proxmox host has 6 NIC's. On NIC1 a trunk comes in with 3 VLAN's. I set my VM's to their specific VLANs. Works great! Buuuut, I want VLAN 2 from NIC1 to be put on NIC2 untagged. How do I do this?
I still don't get what's happening with IPs (internal and external), if this is using NAT or whatever... Are internal machines announced as an independent IP in the range of the bridge?
What's confusing me is having two NICs and each are on a VLAN assigned by the router. But I'm unable to setup a gateway for one of them, despite it need it. Confusing as hell.
I'm a little confused here.. when making a vlan on proxmox, do you need a physical switch for the Vlan nic to work, or is this a virtual switch? Thanks in advance.. and much thanks for the tutorial.
so wait, by bonding 1 cards, can you make it 2 Gbps instead of 1 Gbps or not? imagine Proxmox machine has 2 NICs, each is 1 Gbps, you connect them both to the switch and create aggregation - will every VM have 2 Gbps link to the switch/outer world?
Yes and no. A single TCP session will have only 1Gbps. Multiple sessions in aggregate can add up to 2Gbps. A single file transfer will therefore get only 1Gbps.
You mean, that I can have multiple ISP merged together just like a Loadbalancer? And I could use this proxmox as my point of access to the home network with a LB of ISP? thats sounds great. I thought I had to buy complex and expensive routers and configure them somehow. Something like virtual router... Even tho, I'm not sure if VRRP works as LB(that merge multiple ISP speed connextion).
I am new at proxmox and I will most likely need to look at this video multiple times. So first of all: Thank you. What I have difficulties to grasp at this point is: why would you set an ip address to a bridge? If I should see the bridge as a swithc: a switch does not have an IP address. Is the VM not supposed to set it's own IP address internally, or get one through DHCP? Or is this the fixed IP address, the address for the Proxmox server itself, on this bridge?
Hello. Can i make separate network for VMS like 192.168.10.0/24, and let this vms acces specific services in local network 192.168.1.0/24, such as nas etc. What i want to do is to have spearate network for VMS (K3S cluster) and this cluster should be allowed to acces NFS storage on nas in local network. Also i want to make vms accesible from local network. So i can route traffic to 192.168.10.0/24 I have proxmox installed on intell nuc with only one network card. Can you tell me how to achieve this? Thank you
In general you will need the network's router to handle this. If your existing network is 192.168.1.0/24, you would need to add a static route in the router for the 192.168.10.0/24 subnet via the proxmox host (and configure routing there manually on Linux), or add an additional VLAN on the existing router for 192.168.10.0/24 and use the vlan-aware bridge in Proxmox to forward that to the VMs.
Wow, [mention specific thing you liked about the video]! I especially found [mention specific part you enjoyed] interesting. [Ask a question related to the video]. Keep up the great work! # [relevant hashtag]
This is absolutely outstanding. I read a book with similar content, and it was truly outstanding. "The Art of Meaningful Relationships in the 21st Century" by Leo Flint
Greatly appreciate this video. I've referred back to it several times now when making networking changes to Proxmox. Your examples are very practical, and I'm often hesitant to make networking changes in Proxmox that I'm not completely certain about because I don't want to lose access to the machine. I'm especially thankful that you mentioned the particular use case that Linux VLAN is used for because I needed exactly that feature for my setup. Thanks!
Your 19:22 just saved my ass and I love the fact that you start by saying "linux vlan i very rarely used", turned out that was my missing factor in my infrastructure environment... With this I got full redundancy from my 2 firewalls to my 2 stacked layer 3 switches to my stacked layer 2 switches, which are connected to my 4 host proxmox cluster... I wasn't able to reach the default gateway without the linux vlan tag on the virtuel switch... Thank you so much!
I enjoyed much this video and it was so clear that now I understand well how to take full advantage of these features!!. I also use Mikrotik switches and in my case I had to disable VLAN aware on the vmbr0 as it didn't let the pass traffic or talk with Mikrotik switch. I got IP assignment from DHCP Server but traffic didn't passed through. Disabling the VLAN aware solved my problem!
Absolutely stunning video. Thank you! QQ. If I want to have a VM that is a router using vLANs, is it more efficient to have multiple virtual NICs on the VM with different vLANs tagged in the Proxmox config, or pass it through to a single virtual NIC and then do the tagging on the router? (I hope that makes sense!)
Hello, could you create a tutorial on cluster removal of a node and adding another node to that cluster? also, it would be nice to hace a tutorial on how to improve rdp capabilities of the3 vm so full HD video be played on windows via RDP on proxmox?
Well, maybe a general overview of PCIe devices where IOMMU would be useful. Like maybe (I'm guessing here) IOMMU can be necessary with GPUs used for transcoding or compute; might be useful with some HBA scenarios; and wondering if it could be useful with NICs at all, like when using pfSense OS in a VM.
In general it's needed for PCIe passthrough, be that a GPU, NIC, or HBA. There are other types of passthrough though (bridged NIC, block device, USB) which don't require IOMMU. I'm working on a video on this topic, not pfSense but passthrough methods in general
thanks a lot for that. now i definitely will dive into proxmox again - turned away from it about 2 years ago because of not looking more into the bridge setup also +10 for the mikrotik switch. love their stuff
I enjoyed your video. Regarding proxmox networking in general, what is the best approach to reduce latency? For example, if you're working with video or audio where timing is important.
@@apalrdsadventures Thanks. I was thinking that might be the case, since it would use the asic on the nic. I just started using a Connectx-4 card and it can break out multiple devices for use in SR-IOV. I just need to figure out how to best utilize that functionality across multiple VMs/Containers.
Im trying to follow along but cant seem to get my 2 windows machines to ping each other. I tried using the tag and it didnt work. I have them connected with a bridge and they're still not talking. Not sure what im doing wrong.
Amazing video. Very clear explanations. I started my homelab projects with proxmox, pfsense and etc two years ago but never came across your channel before. However, hats off the way you have made everything clear with examples. I will def. be recommending you to the communities I know.
Best video I've seen on Proxmox network configurations so far. You cover the details that are lacking in many of other videos that led me to this one. 👍
I run pfSense under proxmox and 6 ports on my hardware, only two of which I'm using right now (LAN + WAN). Is there some way I should be configuring them in Proxmox to act collectively as a switch, similar to the one on the LAN side of my ISP router? Or would I pass them all through to pfSense and let that combine them? Note that I do *not* want them on different subnets I just want them to act as switch (or hub). Thanks!
Hi! Thanks for video, very informative. If I may, I'd like to consult my usecase. I recently moved my network management to VMs - I have software router in VM1 and Unifi controller (handling home wifi network - VLAN1 and guest wifi network - VLAN10) on VM2, both in Proxmox. Id like to configure guest network (VLAN10) in Unifi but DHCP server is on VM1 (router). The problem is that Guests cannot get IP adress from router. I am pretty sure that it is the problem with my VLAN config in Proxmox. I gues that router VM should hadnle VLAN1 and 10 same as VM with Unifi. I checked and tagging itself didn't work for me, should I use trunks on both VMs? Any advice will be precious.
If you leave out the VLAN tag, then it will pass all VLANs to the VM. If you want to pass specific VLANs to the VM, you can use the trunks= argument (in the config file, there's no GUI option to set it). Make sure the bridge is set to vlan-aware as well.
So your proxmox server is attached to tagged ports on your switch? How are you setting the vlan ID for vmbr0? Why are you not doing eno1.2, eno1.3 etc and using those for separate bridges vmbr2, vmbr3 etc? If i want to pass the tagged port back into a VM or container how do I do that?
So no need to add vlan devices all the way up. eno1 supports vlan tagging, you can break them out as eno1.2 but don't have to. vmbr bridges (with 'vlan-aware' checked) also support vlan tagging, and this is inherited by child interfaces, so if eno1 is a child of vmbr1 then vlans on eno1 will get passed up to vmbr1 to get processed without creating a bunch of eno1.xx interfaces. Again at vmbr1 we could do vmbr1.2 but again we don't have to. When we create a new network interface on a VM/CT in Proxmox, there's a box to type in the vlan id, and it will essentially make that VM network adapter an access port tagged to that specific vlan id (the id you typed). So the place to configure this is in the VM's hardware, not the host networking. The only exception is if the Proxmox system itself (not the VMs, CTs, the base Linux system) needs to be on a vlan, in that case you'd use a vmbr1.xxx with the IPs set, but you'd never use that for VMs, just the Proxmox base system.
@@apalrdsadventures That's great thank you for your help. I'll give that a go. I've also read I can make a Proxmox workstation in the Proxmox docs by just installing, say, mate or whatever. Perhaps another video for you to do? Perhaps you saw my other comment. I'm getting a lot from your videos. Thank you.
You have access to the full Debian repos on Proxmox. Debian has a few metapackages specifically for installing a full desktop environment on a previously terminal system, they are all named task--desktop. The possibilities are: task-gnome-desktop task-xfce-desktop task-kde-desktop task-lxde-desktop task-cinnamon-desktop task-mate-desktop task-lxqt-desktop `apt install task-xfce-desktop` will install xfce, its basic apps, login manager, and the whole x11 stack. Same for any other desktop. Now you should be careful about messing with things like networking from the desktop, but for a development system it's fine.
Whichever one you want to be primary is the one you put in the GUI. In my case, enx*** is 2.5G and eno1 is 1G, so primary is 2.5G and backup is 1G. But it works just as well with dual 1G.
Thanks for this great Proxmox HowTo. There is one question I still have: Is it possible to receive a Trunk with Proxmox and split the different VLAN of the Trunk to separate Bridges which act like Access Ports? Meaning, that I can simply add a VM to different Bridges to have it connected to the different VLANs?
You can use Linux VLANs off the interface, and set those as the bridge ports on each bridge. Then the bridges are not vlan-aware and only carry traffic for the VLAN of their bridge port.
Isn’t it handy to add vlans 10, 20, 30 interfaces to pve, than add bridges like vmbr10, vmbr20, vmbr30, add vlan interfaces to this bridge, so you can add vm interfaces and assign them to specific bridge which already have pvid 10, 20, 30?
It's more work to create all of the interfaces vs assigning them per VM when you have a decently big deployment. If you use the (experimental) SDN features they can automate a lot of it though. I'm working on a video of SDN with VXLAN.
I have a 5 node cluster setup with HA. Each has one, 1GB NIC that is shared for cluster/VM/PVE GUI. When I attempt to live migrate a VM or Restore from my SAN. I run into the issue where the cluster tires to fence that node. My guess has come down to what you said here. It seems I may need to have a dedicated NIC for corosync. I am assuming the cluster isn't getting the heartbeat do to high latency and is fencing the node. Have you seen this as a problem before?
Hi ! Good informative video. Wanted to know how much reliable is the usb to ethernet. I came to know from the PFSENSE forums, that these power down automatically and cause the link to be down. Let me know your findings on this? It would be helpful
BSD has worse drivers in general than Linux, so you might have a better time with a Linux-based solution than BSD-based if you aren't using very common PCIe NICs.
@@apalrdsadventures I actually managed to do it without VLANs. Proxmox NIC is connected to my physical DMZ port on my router, I created a Linux Bridge (vmbr1) in proxmox that is connected to the physical interface (enp101s0) which becomes the 'WAN' interface for pfSense. I then created another Linux Bridge in proxmox (vmbr2) which is not connected to any physical interface. So, I add vmbr1 and vmbr2 as the two interfaces for the pfsense VM. I then assign them accordingly in pfSense. This seems to work fine without the need for setting up vlans in my home network.
Can you point me to what you used to choose the LACP Bond0 hashing algorithm? I've seen people choose 2+3 and others (like you) choose 3+4 in their guides. I can speculate about it, but in this journey I've embarked on, this is a learning opportunity as much as it is a chance to play with some cool self-hosted homelab stuffs.
Realistically hash based bonds are about the probability of two packets being assigned to different interfaces. So, do you expect to see the most variability in the L2 (MAC), L3 (IP), or L4 (TCP/UDP) headers? In practice it doesn't really matter and the choice is often driven by what hardware offload supports, especially on switches.
A really excellent detail oriented tutorial on networking in the larger sense with Proxmox as the implementation - as well as where to actually *find* all the configuration bits. I've been doing Linux a long time (started with Unix) and this video refreshed and educated me about some networking details that had grown hazy over the years (I let the younger guys deal with it at work 😃). Much appreciated, and subscribed. 👍
![Cody Johnson - I'm Gonna Love You (with Carrie Underwood) [Official Music Video]](http://i.ytimg.com/vi/yy9PuYMU29g/mqdefault.jpg)
Best proxmox' network concepts explanation so far. Good job!
my guy had me at "yo, dwog"
The name's Bond... Bonded Bond
😂😂😂
Shoot, did I miss it? What about bonding tlb and alb? Will watch again. Very informative!
Fantastic. Thank you.
Dude, so much better and clear sense than other Proxmox videos\guides, no forking aroung like any other guide with no clear narration and usefull knowledge.
Much appreciated!
Thanks for the videos. I know some other guys are "more popular" to watch for proxmox but there's nobody that does the depth you do and we really appreciate it.
Yes! And I also appreciate the side notes that you give, just to make sure everybody understands what the terminology is.
Hi, thanks for your video, that´s very interesting and helpful. One question: why is your 2,5 gbit interface marked as half duplex (at 09:32)?
I was able to set up a LAGG in my Proxmox lab using your tutorial first-try (not typical for me). This says a lot about your teaching style. Thanks!
Glad it worked for you!
Thanks for the video! but for your next videos could you please use diagram software to illustrate complex concepts, it definitely helps the community as all other youtubers use it and it's a must in the networking world :)
Excellent video. As someone that started my own homelab and IT journey with 486s in the late 90s and pushed myself ever since, I appreciate you taking the initiative to share this with the community! Gotta keep this stuff in the hands of everyone to learn and build upon it, the "cloud" mentality these days will only destroy what so many have built. Your Ceph on Proxmox video was far more in-depth than one I watched from a large professional outfit (not mentioning any names because they do have a lot of good videos).
Glad you enjoyed it! I definitely like keeping things locally hosted, even if it's just for 'fun'. Hope you enjoy some of the upcoming projects I have!
Agreed, his videos are great! He's doing the community a great service!
Superb. I'm going to have to watch this on slow about 5 times just to get my head around this whole area of Proxmox I knew so little about 🤯 As the song says : "The more I find out, the less I know" 😁 Thankyou.
Glad you like it! This is just the start, there's also the whole Proxmox SDN solution too :)
The imagery of a Linux bridge being "a network switch" and plugging the network interface into it virtually was really helpful! Thanks for that description.
Glad it helped!
Brilliant! Now in my 60's, "homelab-ing" is my new passion, and you made a potentially complex subject look (relatively) easy - thanks! 👍
Glad you enjoyed it!
Do not tuch the stuff in the video if you dont have the keys for the server room at 15:45 on Friday. Don't ask me why and how I know
I’ve watched, what feels like at least, every video on RUclips trying to understand how these concepts work in Proxmox, and this was by far the best and explained every question and issue I had in a single video. Thank you so so so much!
4:06 FINALLY someone on fricking RUclips explains that! I was on several "network videos" about Proxmox before and they didn't explained me that concept of vSwitches like you did THANKS MAN ❤
I appreciate your level of thoroughness on the topic at hand. I also appreciate how you will make mention to other related things to bring about awareness without veering off down a rabbit hole or just omitting them altogether. It's a fine balance and I think you've found it!
Question, have you every connecting a zero client like the EVGA PD05 PCoIP Zero Client
Hi There,
I have 2 dell r720 running pve 8.2.4, connected by 2 1gig links to a cisco 3750 switch. I can get one pve to connect to the switch using lacp for the bond0. the other pve won't connect. same configuration for both bond0s. If you want more info let me know
thx
Hayward
You did a great way of explaining the networking in the most concise way in proxmox, navgiating through the what ifs as well. Ive been kicking myself for days trying to configure vlanning woth ceph and the mgmt ip and so forth. Homelab security is a must.
@apalrdsadventures How do you diagnose issues when a seemingly simple change breaks this? I have trunked VLANs on 1G (pfSense) and a (GS748Tv5) smart switch. I also have a working bonded LACP link between the switch and a NAS, so I am pretty confident the switch is ok.
On proxmox, as soon as I convert the trunk from a NIC port to a bonded NIC (even one), nothing goes through. :( I DID notice that you had to tear it down and build it again to get it to work. I've done that but no joy.
Ideas? THANKS!
Very good video. Thanks!
If you using bonding bonds check that you are not using VLANs on bond0.
I have bond0 (LAGG 10G) and bond1 (backup to 1G). And bond1 not working until I remove all VLANs on bond0.
Thank you!
But I don't understand how to make a access from Inet to my virtual machine, and make my VM isolated from all other my network. Yes, maybe VLANs.... But, o my Gos, am I have to block traffic by all to all (subnets) for every new one subnet (group of virtual machines)?
Great video. Finally sat down to re-do my 10G networking and figured it was time to setup active-backup and vlan awareness. When I did the initial setup, my VMs were fine on another 10G bridge I created but my NFS and iSCSI shares were capped at the 1G speeds - Not anymore! Covered exactly what I needed.
Yo dawg. Nice video 😉
Proxmox GUI has come a long way since v4. Was great to see you showing off the possibilities and no config editing.
I have the task of creating a bond with 10GbE and 1GbE backup, so your video was perfect to help me dry run and visualise how to achieve this without config editing 👍 no doubt this will save me a bunch of time.
You've done a great job of making more advanced network topics accessible to a lot of folks. Bravo.
Glad you liked it! Working on tutorials for some of the more complex parts of the networking GUI (SDN and Firewall)
Thankyou so much for the management vlan trick @20 mins. Cheers
The VLAN are working wehn I don't have VLAN aware checked. As soon as I check it, it quits working. Also if I migrate the VM to the same node in the cluster as teh pfsense, it quits working. I actually have to shut it down, reboot the node that the pfsense vm is running in for it to start working properly again. I've spent many hours today trying to figure out what's wrong lol. I have 3 vlans and native and pfsense is routing it all properly. But as soon as I migrate a vlan tagged VM to the same node as pfsense or set VLAN aware = yes then the doesn't route the traffic.
THANK YOU! This is such a specific thing that is really hard to find instruction on anywhere else. At least that is this detailed.
this is a great tutorial. I have often struggled with this fumbling till it works. The only thing that would have been more helpful is if you went in a little more on the trunk for the vm... I didn't quite follow that.
That was super cool, now you've got me shopping for managed switches so I can get goofy my home network. I've got 4-port cards in all of my infrastructure boxes already...
Managed switches are a ton of fun!
I bonded a bond to a bond and removed the vmbr so now I have to remake the whole thing as I wrecked it again. Whahoo!
Hello, any idea why the 2.5gb interface shows as half-duplex?
why is your 2.5 gig adapter duples at half and your 1g is at full?
lol. that was a nice proxmox video compared to what's out there 👍
Man, I am too nooby to understand all these terms.However, it seems fun to do.
This was quite helpful in configuring proxmox for a pfSense VM that has 3 vlans on a trunk. I missed it in your video, but there was a hint to what I needed to complete the configuration. My host PC has 4x 2.5gb ports, and I wanted to have pfSense serve both the trunk vlans and the local ports with their respective DHCP pools. The bridge was the answer! I was able to bridge the vlan to the local port, with the bridge having the IP address and DHCP server, and the vlan and local port having no IPs.
Glad it helped!
My Proxmox host has 6 NIC's. On NIC1 a trunk comes in with 3 VLAN's. I set my VM's to their specific VLANs. Works great! Buuuut, I want VLAN 2 from NIC1 to be put on NIC2 untagged. How do I do this?
add enx1.2 and enx2 as bridge ports on a non-vlan-aware bridge.
I still don't get what's happening with IPs (internal and external), if this is using NAT or whatever... Are internal machines announced as an independent IP in the range of the bridge?
Proxmox is not involved in IPs other than its own. That's up to the VM.
nice you went extra mile and added 2.5! much appreciated #james bond0
lol thanks! USB NICs aren't ideal, but at least it shows the difference from real 2+G to aggregated 2+G
Thanks for doing this!! Phew...
What's confusing me is having two NICs and each are on a VLAN assigned by the router. But I'm unable to setup a gateway for one of them, despite it need it. Confusing as hell.
You should only have one gateway on a system (unless its a router itself)
This is excellent. Thank you soo so much.
I'm a little confused here.. when making a vlan on proxmox, do you need a physical switch for the Vlan nic to work, or is this a virtual switch? Thanks in advance.. and much thanks for the tutorial.
You do not need a physical switch for VLANs to work. You do need a switch that supports VLANs if you want your vlans to leave the Proxmox box.
so wait, by bonding 1 cards, can you make it 2 Gbps instead of 1 Gbps or not? imagine Proxmox machine has 2 NICs, each is 1 Gbps, you connect them both to the switch and create aggregation - will every VM have 2 Gbps link to the switch/outer world?
Yes and no. A single TCP session will have only 1Gbps. Multiple sessions in aggregate can add up to 2Gbps. A single file transfer will therefore get only 1Gbps.
You mean, that I can have multiple ISP merged together just like a Loadbalancer? And I could use this proxmox as my point of access to the home network with a LB of ISP? thats sounds great. I thought I had to buy complex and expensive routers and configure them somehow. Something like virtual router... Even tho, I'm not sure if VRRP works as LB(that merge multiple ISP speed connextion).
All of this is at layer 2, not layer 3, so no IP routing / load balancing, just MAC.
Great video, thanks man!
You explained a complex subject so simply that even I could understand. Thank you!
Glad it was helpful!
I am new at proxmox and I will most likely need to look at this video multiple times. So first of all: Thank you. What I have difficulties to grasp at this point is: why would you set an ip address to a bridge? If I should see the bridge as a swithc: a switch does not have an IP address. Is the VM not supposed to set it's own IP address internally, or get one through DHCP? Or is this the fixed IP address, the address for the Proxmox server itself, on this bridge?
Setting an IP address on the bridge is essentially plugging in the Proxmox server itself to the bridge, in one step.
Hello.
Can i make separate network for VMS like 192.168.10.0/24, and let this vms acces specific services in local network 192.168.1.0/24, such as nas etc.
What i want to do is to have spearate network for VMS (K3S cluster) and this cluster should be allowed to acces NFS storage on nas in local network. Also i want to make vms accesible from local network. So i can route traffic to 192.168.10.0/24
I have proxmox installed on intell nuc with only one network card.
Can you tell me how to achieve this?
Thank you
In general you will need the network's router to handle this.
If your existing network is 192.168.1.0/24, you would need to add a static route in the router for the 192.168.10.0/24 subnet via the proxmox host (and configure routing there manually on Linux), or add an additional VLAN on the existing router for 192.168.10.0/24 and use the vlan-aware bridge in Proxmox to forward that to the VMs.
Just what the Dr Ordered. Thanks!
Wow, [mention specific thing you liked about the video]! I especially found [mention specific part you enjoyed] interesting. [Ask a question related to the video]. Keep up the great work! # [relevant hashtag]
Ha! I love bondceptions
Just stepping into proxmox with a QNAP TS-470pro converted to pve. This is perfect for helping get the networks setup! Much appreciated.
This is absolutely outstanding. I read a book with similar content, and it was truly outstanding. "The Art of Meaningful Relationships in the 21st Century" by Leo Flint
yup, this cleared up so much in so little time. Thanks for helping on my journey brah.
Happy to help!
Thank for this "advanced" information on Proxmox networking. I am new to Proxmox and I appreciate your video explaining this.
Greatly appreciate this video. I've referred back to it several times now when making networking changes to Proxmox. Your examples are very practical, and I'm often hesitant to make networking changes in Proxmox that I'm not completely certain about because I don't want to lose access to the machine. I'm especially thankful that you mentioned the particular use case that Linux VLAN is used for because I needed exactly that feature for my setup.
Thanks!
Excellent tutorial. Informative, calm style, easy to follow. Simply perfect.
Thank you for this Great video. I managed to setup bond interface on my server just by watching this video and referring to official documentation 😎
Your 19:22 just saved my ass and I love the fact that you start by saying "linux vlan i very rarely used", turned out that was my missing factor in my infrastructure environment...
With this I got full redundancy from my 2 firewalls to my 2 stacked layer 3 switches to my stacked layer 2 switches, which are connected to my 4 host proxmox cluster...
I wasn't able to reach the default gateway without the linux vlan tag on the virtuel switch...
Thank you so much!
I enjoyed much this video and it was so clear that now I understand well how to take full advantage of these features!!. I also use Mikrotik switches and in my case I had to disable VLAN aware on the vmbr0 as it didn't let the pass traffic or talk with Mikrotik switch. I got IP assignment from DHCP Server but traffic didn't passed through. Disabling the VLAN aware solved my problem!
Absolutely stunning video. Thank you!
QQ. If I want to have a VM that is a router using vLANs, is it more efficient to have multiple virtual NICs on the VM with different vLANs tagged in the Proxmox config, or pass it through to a single virtual NIC and then do the tagging on the router? (I hope that makes sense!)
Hello, could you create a tutorial on cluster removal of a node and adding another node to that cluster? also, it would be nice to hace a tutorial on how to improve rdp capabilities of the3 vm so full HD video be played on windows via RDP on proxmox?
Thank you for such a detailed explanation of Proxmox networking.
Have you done a video on NAT in proxmox? Iptables is kicking my ass
I haven't, generally I don't need to use NAT on Proxmox with the Proxmox system bridged to my LAN.
I love this video. Explained a lot. I wonder if you might cover where IOMMU configuration and where it is useful and where it's not.
IOMMU for networking or in general? It's a bit of a different topic
Well, maybe a general overview of PCIe devices where IOMMU would be useful.
Like maybe (I'm guessing here) IOMMU can be necessary with GPUs used for transcoding or compute; might be useful with some HBA scenarios; and wondering if it could be useful with NICs at all, like when using pfSense OS in a VM.
In general it's needed for PCIe passthrough, be that a GPU, NIC, or HBA. There are other types of passthrough though (bridged NIC, block device, USB) which don't require IOMMU.
I'm working on a video on this topic, not pfSense but passthrough methods in general
thanks a lot for that. now i definitely will dive into proxmox again - turned away from it about 2 years ago because of not looking more into the bridge setup
also +10 for the mikrotik switch. love their stuff
I enjoyed your video. Regarding proxmox networking in general, what is the best approach to reduce latency? For example, if you're working with video or audio where timing is important.
That's a good question, and I suspect SR-IOV will get you the least jitter as the software bridge will be more dependent on CPU load.
@@apalrdsadventures Thanks. I was thinking that might be the case, since it would use the asic on the nic. I just started using a Connectx-4 card and it can break out multiple devices for use in SR-IOV. I just need to figure out how to best utilize that functionality across multiple VMs/Containers.
Only just come across your channel and I'm hooked, keep up the great work and a tickle under the chin to Sherlock!
Im trying to follow along but cant seem to get my 2 windows machines to ping each other. I tried using the tag and it didnt work. I have them connected with a bridge and they're still not talking. Not sure what im doing wrong.
Amazing video. Very clear explanations. I started my homelab projects with proxmox, pfsense and etc two years ago but never came across your channel before. However, hats off the way you have made everything clear with examples. I will def. be recommending you to the communities I know.
The most concise proxmox networking, and linux in general. Thanks
Excellent Analysis!
The perfect deep level of detail I was looking for. your vids are amazing.
Best video I've seen on Proxmox network configurations so far. You cover the details that are lacking in many of other videos that led me to this one. 👍
Glad you like it!
Top notch. Very detailed and informative. Thank you!
Great video. I do wish you could have gone down the cluster rabbit hole a bit. I'd like to see how that gets setup.
I run pfSense under proxmox and 6 ports on my hardware, only two of which I'm using right now (LAN + WAN). Is there some way I should be configuring them in Proxmox to act collectively as a switch, similar to the one on the LAN side of my ISP router? Or would I pass them all through to pfSense and let that combine them? Note that I do *not* want them on different subnets I just want them to act as switch (or hub). Thanks!
You can assign multiple ports to a bridge (vmbr) and it will act like a switch.
@@apalrdsadventures Thanks for getting back to me! I can't believe it's that simple, I should have just tried that, it works great.
Excellent video.. If you can please post a video with evpn vxlan in proxmox. Great video!!!
I just did SDN basics, so it will be next in the SDN list (unicast vxlan and evpn vxlan)
@@apalrdsadventures Fantastic!! Hope soon because SDN is very very great technology in Proxmox. Thanks!!
Magic 🪄 what a wonderful video. This needs to go viral.
Thanks for the demo and info, now my proxmox is speedier! Have a great day
ola, estou com problema de subir a latencia nas VMs quando vou copiar um arquivo grande, você ja passou por isso
Thanks for this brilliant video!
6:25 did you mean VLAN tag, in stead of VM tag?
oh yeah I did
Thanks a lot!
Hi! Thanks for video, very informative. If I may, I'd like to consult my usecase. I recently moved my network management to VMs - I have software router in VM1 and Unifi controller (handling home wifi network - VLAN1 and guest wifi network - VLAN10) on VM2, both in Proxmox. Id like to configure guest network (VLAN10) in Unifi but DHCP server is on VM1 (router). The problem is that Guests cannot get IP adress from router. I am pretty sure that it is the problem with my VLAN config in Proxmox. I gues that router VM should hadnle VLAN1 and 10 same as VM with Unifi. I checked and tagging itself didn't work for me, should I use trunks on both VMs? Any advice will be precious.
If you leave out the VLAN tag, then it will pass all VLANs to the VM. If you want to pass specific VLANs to the VM, you can use the trunks= argument (in the config file, there's no GUI option to set it). Make sure the bridge is set to vlan-aware as well.
So your proxmox server is attached to tagged ports on your switch? How are you setting the vlan ID for vmbr0? Why are you not doing eno1.2, eno1.3 etc and using those for separate bridges vmbr2, vmbr3 etc? If i want to pass the tagged port back into a VM or container how do I do that?
So no need to add vlan devices all the way up. eno1 supports vlan tagging, you can break them out as eno1.2 but don't have to. vmbr bridges (with 'vlan-aware' checked) also support vlan tagging, and this is inherited by child interfaces, so if eno1 is a child of vmbr1 then vlans on eno1 will get passed up to vmbr1 to get processed without creating a bunch of eno1.xx interfaces.
Again at vmbr1 we could do vmbr1.2 but again we don't have to. When we create a new network interface on a VM/CT in Proxmox, there's a box to type in the vlan id, and it will essentially make that VM network adapter an access port tagged to that specific vlan id (the id you typed). So the place to configure this is in the VM's hardware, not the host networking.
The only exception is if the Proxmox system itself (not the VMs, CTs, the base Linux system) needs to be on a vlan, in that case you'd use a vmbr1.xxx with the IPs set, but you'd never use that for VMs, just the Proxmox base system.
@@apalrdsadventures That's great thank you for your help. I'll give that a go. I've also read I can make a Proxmox workstation in the Proxmox docs by just installing, say, mate or whatever. Perhaps another video for you to do? Perhaps you saw my other comment. I'm getting a lot from your videos. Thank you.
You have access to the full Debian repos on Proxmox. Debian has a few metapackages specifically for installing a full desktop environment on a previously terminal system, they are all named task--desktop. The possibilities are: task-gnome-desktop task-xfce-desktop task-kde-desktop task-lxde-desktop task-cinnamon-desktop task-mate-desktop task-lxqt-desktop
`apt install task-xfce-desktop` will install xfce, its basic apps, login manager, and the whole x11 stack. Same for any other desktop.
Now you should be careful about messing with things like networking from the desktop, but for a development system it's fine.
Thank you for very useful video, if active-backup mode which device should be primary or backup in the GUI : eno1 or enx0024278832fb device ?
Whichever one you want to be primary is the one you put in the GUI. In my case, enx*** is 2.5G and eno1 is 1G, so primary is 2.5G and backup is 1G. But it works just as well with dual 1G.
Thanks for this great Proxmox HowTo. There is one question I still have: Is it possible to receive a Trunk with Proxmox and split the different VLAN of the Trunk to separate Bridges which act like Access Ports? Meaning, that I can simply add a VM to different Bridges to have it connected to the different VLANs?
You can use Linux VLANs off the interface, and set those as the bridge ports on each bridge. Then the bridges are not vlan-aware and only carry traffic for the VLAN of their bridge port.
Thank you!
Isn’t it handy to add vlans 10, 20, 30 interfaces to pve, than add bridges like vmbr10, vmbr20, vmbr30, add vlan interfaces to this bridge, so you can add vm interfaces and assign them to specific bridge which already have pvid 10, 20, 30?
It's more work to create all of the interfaces vs assigning them per VM when you have a decently big deployment.
If you use the (experimental) SDN features they can automate a lot of it though. I'm working on a video of SDN with VXLAN.
came from discord
Thanks for the explanations!!!!
Nice work
I have a 5 node cluster setup with HA. Each has one, 1GB NIC that is shared for cluster/VM/PVE GUI. When I attempt to live migrate a VM or Restore from my SAN. I run into the issue where the cluster tires to fence that node. My guess has come down to what you said here. It seems I may need to have a dedicated NIC for corosync. I am assuming the cluster isn't getting the heartbeat do to high latency and is fencing the node. Have you seen this as a problem before?
It could potentially be. I haven't seen it fence a node that was running properly, but I'm also not pushing the network super hard in testing
Hi ! Good informative video. Wanted to know how much reliable is the usb to ethernet. I came to know from the PFSENSE forums, that these power down automatically and cause the link to be down. Let me know your findings on this? It would be helpful
BSD has worse drivers in general than Linux, so you might have a better time with a Linux-based solution than BSD-based if you aren't using very common PCIe NICs.
How would I go about adding a WAN and LAN interface for a virtualised instance of pfSense if I have just one physical interface?
You'd need a switch which can split out VLAN tags
@@apalrdsadventures I actually managed to do it without VLANs. Proxmox NIC is connected to my physical DMZ port on my router, I created a Linux Bridge (vmbr1) in proxmox that is connected to the physical interface (enp101s0) which becomes the 'WAN' interface for pfSense. I then created another Linux Bridge in proxmox (vmbr2) which is not connected to any physical interface. So, I add vmbr1 and vmbr2 as the two interfaces for the pfsense VM. I then assign them accordingly in pfSense. This seems to work fine without the need for setting up vlans in my home network.
That works if you don't want to share the LAN interface outside of the Proxmox system, but the external router is also doing NAT in this case.
Can you point me to what you used to choose the LACP Bond0 hashing algorithm? I've seen people choose 2+3 and others (like you) choose 3+4 in their guides.
I can speculate about it, but in this journey I've embarked on, this is a learning opportunity as much as it is a chance to play with some cool self-hosted homelab stuffs.
Realistically hash based bonds are about the probability of two packets being assigned to different interfaces. So, do you expect to see the most variability in the L2 (MAC), L3 (IP), or L4 (TCP/UDP) headers? In practice it doesn't really matter and the choice is often driven by what hardware offload supports, especially on switches.
A really excellent detail oriented tutorial on networking in the larger sense with Proxmox as the implementation - as well as where to actually *find* all the configuration bits. I've been doing Linux a long time (started with Unix) and this video refreshed and educated me about some networking details that had grown hazy over the years (I let the younger guys deal with it at work 😃).
Much appreciated, and subscribed. 👍
Glad it was helpful!