I have passed through a two port nic to my virtulized PFSense vm. That gives me enough feeling of a physical separation to keep me happy and still able to consolidate some hardware.
I started out virtualising it, and I still have a pfsense VM as a backup, but I just prefer to have it on bare metal now. Just less headaches, but it's completely a viable option if you want to make the most out of your hardware and are comfortable with the internet going down upon a reboot (without HA), or you just want to tinker. EDIT: Ohh and snapshots can make your life less miserable with pfsense CE if you mess up. pfsense Plus has boot environments to revert back to.
Same setup here. With no HA, my proxmox system is rebooted way more often than my physical firewall. Actually my physical has never been rebooted except for updates. And having a virtualized backup allows you to test updates and configs prior to deploying to the physical. Ideal setup imo.
Two things I'll never virtualise - storage and firewalls. I know its a luxury to not HAVE to do this, but I won't put either of those two crucial functions on the same platform as I am testing other software and features on, and will be up and down sporadically.
I used to be the same, but like Jim, once I got a hypervisor cluster, I gladly moved my firewall from physical to virtual and have zero regrets. This change has made my firewall more available than when on dedicated hardware.
If you encounter errors during the initial boot ("BdsDxe: failed to load Boot0003 "UEFI PXEv4" from PciRoot" and similar) please uncheck "Pre-Enroll Keys" field in EFI disk creation box.
One of the reasons I got the MS-01 was with virtualizing Opnsense in mind, but really, I haven't tried it, as I didn't want my whole network to go down if I needed to update Proxmox or work on the hardware or something. I feel like virtualizing it only makes sense if you have a Proxmox cluster with HA so one of the other nodes can take over in that instance.
I'm running all my VMs on i440fx mostly SeaBIOS, passthrough two NICs on OPNsense VM and passthrough a Zigbee Dongle on Homeassistant VM without any problems. 🧐
@@Jims-Garage Just read "PCIe passthrough is only available on q35 machines". Damn, didnt know that, I just started with Proxmox without any guides and went the default route. So maybe I'll be lucky and nothing will ever happen 😁
I've done pfsense virtualized for a couple of years, on bare metal for a couple of years and now I'm back virtualizing pfsense. As long as it runs both options are fine, but about a year ago a pfsense update killed my bare metal installation as the boot partition wasn't big enough for the update. For some reason the pfsence update function didn't bother to check if the boot partition was big enough for the update, so it crashed during update and left me with a miserable afternoon of trying to fix the install without internet access. So now I'm back to running pfsence virtualized so I can snapshot and roll back if an update breaks pfsense. I really struggle to see the benefit of running pfsense on bare metal unless you have very minimal hardware. Even a thin client system like the Dell Wyse 5070 is plenty strong enough to run pfsense as a vm under proxmox, so why not get the benefit of doing backups of the pfsense vm, snapshots before updates, the ability to move the vm to another server while updating hardware and all that?
Simplicity and high availability. To my knowledge you cannot do migration or backups with hardware passthrough. I'm still able to max out my bandwidth with a VMBR.
Thanks for the video. I see you have Firewall=1 on the 2 network devices, is that required? My pfsense vm always stops at startup with some errors and the last line says masks and some hex code. I can stop the vm and then do a restore from my PBS and voila it boots all the way. Edit: Added a Serial Port in Hardware and checked that EFI - Attempt Secure Boot is unchecked.
Still having a dilemma regarding nics. Intel 2.5GB being buggy, 10GB being expensive for my use case. Is there any advice regarding 2.5GB? Thanks in advance! My current firewall setup is with the use of vlans with single gigabit nic with help of opnsense vids from jim
You do not need q35 for pfSense, it function just fine with the default. Also you cannot say that vmbr0 is vnet0 or vnet1 until you actually have looked it at - which is probably the most annoying part of the pfSense installation (I do not know opnsense).
@@casperghst42 I know, I said that in the video (q35 is only required for passthrough). Once you create the vmbr you only need to look at the MAC address to understand which is which, another benefit of a VM.
Moving my bare-metal install to VM this week, very coincidental. Mainly because as an edge device, the FreeBSD base system is problematic and difficult to maintain. Running VM it's easier to add other edge processes/features without mucking up pfSense. Small host with 2x 10G and 4x 2.5G NICs But there's no way I'm doing it with ProxMox which is a horrible platform and UI for KVM. pfSense easy enough, but LXCs are especially where it falls down.
The timing of this video is unreal - I am debating this myself at this very moment. People who virtualize your firewall, are you running a physical firewall before your whole home network? Additionally, are you exposing any services to the internet?
@@jacobnoori ISP router (technically SFP modem) straight into the virtual firewall (via a dedicated switch that splits internet across all nodes - necessary for failover)
I use the default vmbr0 as my LAN connection. Not the best practice but it works fine. Ideally you'd want your node's host IP bound to a different VLAN but I haven't gotten that far.
We all love details explanation,Well Done Jim!
Thanks 👍
I have passed through a two port nic to my virtulized PFSense vm. That gives me enough feeling of a physical separation to keep me happy and still able to consolidate some hardware.
I actually do half and half. The box I run PfSense on only has two nics, so I pass one through as hardware for WAN and then use a virtual nic for LAN.
I'm doing the same. This setup is convenient and easy to work with
nice to see your videoclips. thanks.
I started out virtualising it, and I still have a pfsense VM as a backup, but I just prefer to have it on bare metal now. Just less headaches, but it's completely a viable option if you want to make the most out of your hardware and are comfortable with the internet going down upon a reboot (without HA), or you just want to tinker.
EDIT: Ohh and snapshots can make your life less miserable with pfsense CE if you mess up. pfsense Plus has boot environments to revert back to.
Same setup here. With no HA, my proxmox system is rebooted way more often than my physical firewall. Actually my physical has never been rebooted except for updates. And having a virtualized backup allows you to test updates and configs prior to deploying to the physical. Ideal setup imo.
This video motivated me to actually document and label my NIC port locations.
It's so helpful at exactly the times you need it!
@@Jims-Garage It really is. I think I was just in a hurry to set things up before, but documentation and labels are the way forward.
Nice, thanks for sharing this
Two things I'll never virtualise - storage and firewalls. I know its a luxury to not HAVE to do this, but I won't put either of those two crucial functions on the same platform as I am testing other software and features on, and will be up and down sporadically.
@@gaidin makes sense if you only have a single node, I used to do the same until I had a cluster
I used to be the same, but like Jim, once I got a hypervisor cluster, I gladly moved my firewall from physical to virtual and have zero regrets. This change has made my firewall more available than when on dedicated hardware.
never heard of SD wan then huh
If you encounter errors during the initial boot ("BdsDxe: failed to load Boot0003 "UEFI PXEv4" from PciRoot" and similar) please uncheck "Pre-Enroll Keys" field in EFI disk creation box.
One of the reasons I got the MS-01 was with virtualizing Opnsense in mind, but really, I haven't tried it, as I didn't want my whole network to go down if I needed to update Proxmox or work on the hardware or something. I feel like virtualizing it only makes sense if you have a Proxmox cluster with HA so one of the other nodes can take over in that instance.
I'm running all my VMs on i440fx mostly SeaBIOS, passthrough two NICs on OPNsense VM and passthrough a Zigbee Dongle on Homeassistant VM without any problems. 🧐
@@Colebrath nice, check the Proxmox docs though, it's only officially supported on q35. Hopefully an update doesn't break it.
@@Jims-Garage Just read "PCIe passthrough is only available on q35 machines". Damn, didnt know that, I just started with Proxmox without any guides and went the default route. So maybe I'll be lucky and nothing will ever happen 😁
I've done pfsense virtualized for a couple of years, on bare metal for a couple of years and now I'm back virtualizing pfsense. As long as it runs both options are fine, but about a year ago a pfsense update killed my bare metal installation as the boot partition wasn't big enough for the update. For some reason the pfsence update function didn't bother to check if the boot partition was big enough for the update, so it crashed during update and left me with a miserable afternoon of trying to fix the install without internet access. So now I'm back to running pfsence virtualized so I can snapshot and roll back if an update breaks pfsense.
I really struggle to see the benefit of running pfsense on bare metal unless you have very minimal hardware. Even a thin client system like the Dell Wyse 5070 is plenty strong enough to run pfsense as a vm under proxmox, so why not get the benefit of doing backups of the pfsense vm, snapshots before updates, the ability to move the vm to another server while updating hardware and all that?
Is there a reason you aren't using SR-IOV with your intel x710?
Simplicity and high availability. To my knowledge you cannot do migration or backups with hardware passthrough. I'm still able to max out my bandwidth with a VMBR.
Thanks for the video. I see you have Firewall=1 on the 2 network devices, is that required? My pfsense vm always stops at startup with some errors and the last line says masks and some hex code. I can stop the vm and then do a restore from my PBS and voila it boots all the way. Edit: Added a Serial Port in Hardware and checked that EFI - Attempt Secure Boot is unchecked.
Still having a dilemma regarding nics. Intel 2.5GB being buggy, 10GB being expensive for my use case. Is there any advice regarding 2.5GB? Thanks in advance! My current firewall setup is with the use of vlans with single gigabit nic with help of opnsense vids from jim
Mellanox connectx-3 10Gb are pretty cheap on eBay.
You do not need q35 for pfSense, it function just fine with the default. Also you cannot say that vmbr0 is vnet0 or vnet1 until you actually have looked it at - which is probably the most annoying part of the pfSense installation (I do not know opnsense).
@@casperghst42 I know, I said that in the video (q35 is only required for passthrough). Once you create the vmbr you only need to look at the MAC address to understand which is which, another benefit of a VM.
@@Jims-Garage I’ve run my pfSense with one nic passthrough where I didn’t use q35. This is on Gen8 cpus.
Moving my bare-metal install to VM this week, very coincidental. Mainly because as an edge device, the FreeBSD base system is problematic and difficult to maintain. Running VM it's easier to add other edge processes/features without mucking up pfSense. Small host with 2x 10G and 4x 2.5G NICs
But there's no way I'm doing it with ProxMox which is a horrible platform and UI for KVM. pfSense easy enough, but LXCs are especially where it falls down.
This is a hot take. In my experience I’ve had the opposite experience with Proxmox containers.
i run virtual pfSense on a single node. im crazy that way....
@@brachisaurous livin' on the edge!
@@Jims-Garage a wise man recently told me...edge computing is the future!
Didn't you just have the same setup with opnsense.
Did not had the time to whatch the whole video, but i am curious.
I did, but people wanted me to cover pfSense. Plus it's a good opportunity to check it out, I like to stay abreast of developments.
@@Jims-Garage Thank you for explaining, I will watch the video now, I'm sure it is going to be great!
The timing of this video is unreal - I am debating this myself at this very moment. People who virtualize your firewall, are you running a physical firewall before your whole home network? Additionally, are you exposing any services to the internet?
@@jacobnoori ISP router (technically SFP modem) straight into the virtual firewall (via a dedicated switch that splits internet across all nodes - necessary for failover)
@@Jims-Garage Thanks, Jim. I see you have a video on that which I plan on watching. Do you utilize the PVE firewall?
How about a vid on opnsense bare metal install. Ta!
@@talismanna check the first video on my OPNSense series (there's a playlist)
How can I do this with two nics but I still want to access the node?
I use the default vmbr0 as my LAN connection. Not the best practice but it works fine. Ideally you'd want your node's host IP bound to a different VLAN but I haven't gotten that far.
For me it's Asus router, I don't have energy to play with hardware😆