I just found these videos as I am looking to expand my basic homelad that I have for hobby purposes. Loving the content and the 40k references in your proxmox set up XD
Very helpful. I so want to do stuff like you've got set up. Maybe, over the Holidays, I'll have time to stop Life and have some "me time" to be able to play with this & get everything going. > Yes, do cover the features of PfSense ... 16:00 ... because I think that would help a lot of people that are just waking up to realizing that they don't have to use Xfinity's hardware anymore.
I have passed through a two port nic to my virtulized PFSense vm. That gives me enough feeling of a physical separation to keep me happy and still able to consolidate some hardware.
If you encounter errors during the initial boot ("BdsDxe: failed to load Boot0003 "UEFI PXEv4" from PciRoot" and similar) please uncheck "Pre-Enroll Keys" field in EFI disk creation box.
One of the reasons I got the MS-01 was with virtualizing Opnsense in mind, but really, I haven't tried it, as I didn't want my whole network to go down if I needed to update Proxmox or work on the hardware or something. I feel like virtualizing it only makes sense if you have a Proxmox cluster with HA so one of the other nodes can take over in that instance.
I started out virtualising it, and I still have a pfsense VM as a backup, but I just prefer to have it on bare metal now. Just less headaches, but it's completely a viable option if you want to make the most out of your hardware and are comfortable with the internet going down upon a reboot (without HA), or you just want to tinker. EDIT: Ohh and snapshots can make your life less miserable with pfsense CE if you mess up. pfsense Plus has boot environments to revert back to.
Same setup here. With no HA, my proxmox system is rebooted way more often than my physical firewall. Actually my physical has never been rebooted except for updates. And having a virtualized backup allows you to test updates and configs prior to deploying to the physical. Ideal setup imo.
I've done pfsense virtualized for a couple of years, on bare metal for a couple of years and now I'm back virtualizing pfsense. As long as it runs both options are fine, but about a year ago a pfsense update killed my bare metal installation as the boot partition wasn't big enough for the update. For some reason the pfsence update function didn't bother to check if the boot partition was big enough for the update, so it crashed during update and left me with a miserable afternoon of trying to fix the install without internet access. So now I'm back to running pfsence virtualized so I can snapshot and roll back if an update breaks pfsense. I really struggle to see the benefit of running pfsense on bare metal unless you have very minimal hardware. Even a thin client system like the Dell Wyse 5070 is plenty strong enough to run pfsense as a vm under proxmox, so why not get the benefit of doing backups of the pfsense vm, snapshots before updates, the ability to move the vm to another server while updating hardware and all that?
Thanks for the video. I see you have Firewall=1 on the 2 network devices, is that required? My pfsense vm always stops at startup with some errors and the last line says masks and some hex code. I can stop the vm and then do a restore from my PBS and voila it boots all the way. Edit: Added a Serial Port in Hardware and checked that EFI - Attempt Secure Boot is unchecked.
Simplicity and high availability. To my knowledge you cannot do migration or backups with hardware passthrough. I'm still able to max out my bandwidth with a VMBR.
Still having a dilemma regarding nics. Intel 2.5GB being buggy, 10GB being expensive for my use case. Is there any advice regarding 2.5GB? Thanks in advance! My current firewall setup is with the use of vlans with single gigabit nic with help of opnsense vids from jim
I'm running all my VMs on i440fx mostly SeaBIOS, passthrough two NICs on OPNsense VM and passthrough a Zigbee Dongle on Homeassistant VM without any problems. 🧐
@@Jims-Garage Just read "PCIe passthrough is only available on q35 machines". Damn, didnt know that, I just started with Proxmox without any guides and went the default route. So maybe I'll be lucky and nothing will ever happen 😁
I use the default vmbr0 as my LAN connection. Not the best practice but it works fine. Ideally you'd want your node's host IP bound to a different VLAN but I haven't gotten that far.
You do not need q35 for pfSense, it function just fine with the default. Also you cannot say that vmbr0 is vnet0 or vnet1 until you actually have looked it at - which is probably the most annoying part of the pfSense installation (I do not know opnsense).
@@casperghst42 I know, I said that in the video (q35 is only required for passthrough). Once you create the vmbr you only need to look at the MAC address to understand which is which, another benefit of a VM.
Moving my bare-metal install to VM this week, very coincidental. Mainly because as an edge device, the FreeBSD base system is problematic and difficult to maintain. Running VM it's easier to add other edge processes/features without mucking up pfSense. Small host with 2x 10G and 4x 2.5G NICs But there's no way I'm doing it with ProxMox which is a horrible platform and UI for KVM. pfSense easy enough, but LXCs are especially where it falls down.
The timing of this video is unreal - I am debating this myself at this very moment. People who virtualize your firewall, are you running a physical firewall before your whole home network? Additionally, are you exposing any services to the internet?
@@jacobnoori ISP router (technically SFP modem) straight into the virtual firewall (via a dedicated switch that splits internet across all nodes - necessary for failover)
I just found these videos as I am looking to expand my basic homelad that I have for hobby purposes. Loving the content and the 40k references in your proxmox set up XD
Really appreciate the feedback
We all love details explanation,Well Done Jim!
Thanks 👍
nice to see your videoclips. thanks.
Nice, thanks for sharing this
This video motivated me to actually document and label my NIC port locations.
It's so helpful at exactly the times you need it!
@@Jims-Garage It really is. I think I was just in a hurry to set things up before, but documentation and labels are the way forward.
Very helpful. I so want to do stuff like you've got set up. Maybe, over the Holidays, I'll have time to stop Life and have some "me time" to be able to play with this & get everything going.
> Yes, do cover the features of PfSense ... 16:00 ... because I think that would help a lot of people that are just waking up to realizing that they don't have to use Xfinity's hardware anymore.
I have passed through a two port nic to my virtulized PFSense vm. That gives me enough feeling of a physical separation to keep me happy and still able to consolidate some hardware.
I actually do half and half. The box I run PfSense on only has two nics, so I pass one through as hardware for WAN and then use a virtual nic for LAN.
I'm doing the same. This setup is convenient and easy to work with
If you encounter errors during the initial boot ("BdsDxe: failed to load Boot0003 "UEFI PXEv4" from PciRoot" and similar) please uncheck "Pre-Enroll Keys" field in EFI disk creation box.
One of the reasons I got the MS-01 was with virtualizing Opnsense in mind, but really, I haven't tried it, as I didn't want my whole network to go down if I needed to update Proxmox or work on the hardware or something. I feel like virtualizing it only makes sense if you have a Proxmox cluster with HA so one of the other nodes can take over in that instance.
I started out virtualising it, and I still have a pfsense VM as a backup, but I just prefer to have it on bare metal now. Just less headaches, but it's completely a viable option if you want to make the most out of your hardware and are comfortable with the internet going down upon a reboot (without HA), or you just want to tinker.
EDIT: Ohh and snapshots can make your life less miserable with pfsense CE if you mess up. pfsense Plus has boot environments to revert back to.
Same setup here. With no HA, my proxmox system is rebooted way more often than my physical firewall. Actually my physical has never been rebooted except for updates. And having a virtualized backup allows you to test updates and configs prior to deploying to the physical. Ideal setup imo.
I've done pfsense virtualized for a couple of years, on bare metal for a couple of years and now I'm back virtualizing pfsense. As long as it runs both options are fine, but about a year ago a pfsense update killed my bare metal installation as the boot partition wasn't big enough for the update. For some reason the pfsence update function didn't bother to check if the boot partition was big enough for the update, so it crashed during update and left me with a miserable afternoon of trying to fix the install without internet access. So now I'm back to running pfsence virtualized so I can snapshot and roll back if an update breaks pfsense.
I really struggle to see the benefit of running pfsense on bare metal unless you have very minimal hardware. Even a thin client system like the Dell Wyse 5070 is plenty strong enough to run pfsense as a vm under proxmox, so why not get the benefit of doing backups of the pfsense vm, snapshots before updates, the ability to move the vm to another server while updating hardware and all that?
Plz can you tell me, how can I access pfsense web GUI from windows machine which is in (VTNET1) on proxmox?
Assuming default config it should be on 192.168.1.1
Thanks for the video. I see you have Firewall=1 on the 2 network devices, is that required? My pfsense vm always stops at startup with some errors and the last line says masks and some hex code. I can stop the vm and then do a restore from my PBS and voila it boots all the way. Edit: Added a Serial Port in Hardware and checked that EFI - Attempt Secure Boot is unchecked.
Is there a reason you aren't using SR-IOV with your intel x710?
Simplicity and high availability. To my knowledge you cannot do migration or backups with hardware passthrough. I'm still able to max out my bandwidth with a VMBR.
why qemu agent disabled ?
@@szerokootwarteoczy I tend to install in the VM first and then retrospectively enable it.
Still having a dilemma regarding nics. Intel 2.5GB being buggy, 10GB being expensive for my use case. Is there any advice regarding 2.5GB? Thanks in advance! My current firewall setup is with the use of vlans with single gigabit nic with help of opnsense vids from jim
Mellanox connectx-3 10Gb are pretty cheap on eBay.
I'm running all my VMs on i440fx mostly SeaBIOS, passthrough two NICs on OPNsense VM and passthrough a Zigbee Dongle on Homeassistant VM without any problems. 🧐
@@Colebrath nice, check the Proxmox docs though, it's only officially supported on q35. Hopefully an update doesn't break it.
@@Jims-Garage Just read "PCIe passthrough is only available on q35 machines". Damn, didnt know that, I just started with Proxmox without any guides and went the default route. So maybe I'll be lucky and nothing will ever happen 😁
How can I do this with two nics but I still want to access the node?
I use the default vmbr0 as my LAN connection. Not the best practice but it works fine. Ideally you'd want your node's host IP bound to a different VLAN but I haven't gotten that far.
You do not need q35 for pfSense, it function just fine with the default. Also you cannot say that vmbr0 is vnet0 or vnet1 until you actually have looked it at - which is probably the most annoying part of the pfSense installation (I do not know opnsense).
@@casperghst42 I know, I said that in the video (q35 is only required for passthrough). Once you create the vmbr you only need to look at the MAC address to understand which is which, another benefit of a VM.
@@Jims-Garage I’ve run my pfSense with one nic passthrough where I didn’t use q35. This is on Gen8 cpus.
Moving my bare-metal install to VM this week, very coincidental. Mainly because as an edge device, the FreeBSD base system is problematic and difficult to maintain. Running VM it's easier to add other edge processes/features without mucking up pfSense. Small host with 2x 10G and 4x 2.5G NICs
But there's no way I'm doing it with ProxMox which is a horrible platform and UI for KVM. pfSense easy enough, but LXCs are especially where it falls down.
This is a hot take. In my experience I’ve had the opposite experience with Proxmox containers.
i run virtual pfSense on a single node. im crazy that way....
@@brachisaurous livin' on the edge!
@@Jims-Garage a wise man recently told me...edge computing is the future!
The timing of this video is unreal - I am debating this myself at this very moment. People who virtualize your firewall, are you running a physical firewall before your whole home network? Additionally, are you exposing any services to the internet?
@@jacobnoori ISP router (technically SFP modem) straight into the virtual firewall (via a dedicated switch that splits internet across all nodes - necessary for failover)
@@Jims-Garage Thanks, Jim. I see you have a video on that which I plan on watching. Do you utilize the PVE firewall?
For me it's Asus router, I don't have energy to play with hardware😆