Which firewall / router are you running at home? If you can't remember, maybe it's time to SWITCH ;) By the way, if you're new here, welcome! Please remember to ✨subscribe✨ for more content like this!
Used Pfsense since 3 years back inside of a proxmox machine at home. Coupled together are a XCP-NG machine running Pfsense at my parents house with IPsec tunneling. (150km) Getting 250/250mbit sym bandwith between the two places. :)
@@TechnoTim I think both are realy good products. XCP-ng do have a easier time to bind a swarm of servers in my opinion. And i do like the disaster recovery, XenMotion, True backup system compare to tar.gz of proxmox. But as i said. You can't go wrong with either.
just a question, but is it safe to use qnap nas virtual station for pfsense vm? compared to using say a nuc install ubuntu? then install proxmox? then install pfsense freebsd as a vm using that proxmox? no portforwarding being done, just merely using as an edge router firewall.
I have an identical setup. One thing to consider depending on how many cores you have on the host, is to make the CPU type 'host' and pass through 1 or 2 physical cores. This should ( depending on your CPU ) enable the AES-NI CPU crypto which can be useful if you use OpenVPN and want faster throughput over encrypted connections. Awesome guides by the way, I wish these vids were around years ago!
Heyo Tim, you have greatly helped me get into the Homelab scene, and I appreciate it. With that said, you really should consider revisiting this video with a 2022/2023 edition. Reason why I say this is because passing my NIC down to the OPNSense VM in Proxmox (and even Pfsense) straight up did not work. I almost gave up, until I talked to someone that had a workaround: by creating a Linux bridge with the NIC as an alternative way. Passing the NICs down did not work but creating a bridge did. I had other people express their grievance about following your video and having it not work. And from what I heard, when it comes to virtualizing routers/firewalls, passing down NICs is a huge NoNo for this reason. I have no doubt this worked for some people, but I feel like there is a higher chance of success with an updated video by using the create Linux bridge method. Just my 2 cents!
He recommended the first method likely because it maximizes performance to just pass through, but bridging through a virtual interface works likely as well. His suggestion is ideal when supported. If not, with VM's you can typically emulate a method instead. Pass through in general can get finicky for some.
Such a great idea for those tech heads that want to do something more than what those basic modem routers.. Just a note for those with different NBN connections that you may still need the netgear/gateway/modem from your ISP but simply put it into bridge mode then pass that to the WAN interface as per TechnoTim's guide!! (suit most Australian NBN type of setups) As I am and Aussie viewer also!!!
Thank you for this video. I have one “noob”question. Using a physical machine that has 6 network ports, running ProxMox and a pfSense VM...how can I access ProxMox web control panel from my network that is being served by pfSense? Do I just need to ensure ProxMox is on the same subnet as my LAN? Thank you kindly for helping.
I am wondering how you interact with Proxmox after you virtualize your network as a VM through Proxmox... I am wondering how the system determines an IP through a VM that hasn't booted yet. After it boots, how does it get an IP from the VM?
Basically you always want to use host, unless you want to do a live migration to a different host with a different cpu. Kvm has the bare minimum of cpu flags, host type is always better
@@succubiuseisspin3707 precisely for live vm migration reasons. I learned that the hard way when I couldn't figure out why my tls offloading proxy was so slow on my proxmox vm... Then I dug around and found some official docs covering that.
Hi Tim fantastic video! I'm just getting started with Proxmox but so far I am digging it, I want to set up a virtual PFsense instance but not to act as my real firewall in my office, I just want to be able to join other VM’s within Proxmox to the LAN network that PFsense is creating. That way I could test VPN solutions like Wireguard, Zerotier and Open VPN from one VM to another that are on different networks. My Proxmox box does have 2 NICS, actually 3, what would be the best way to go about this? I feel like I can basically follow your tutorial except for on the LAN NIC for PF sense I don't need to connect it to a switch I just need it to broadcast to the other VMS in Proxmox, just not quite sure how to do that. Thanks !
Can I ask a question, probably a stupid one but here goes... How are the two wired? I have my internet coming into my home to my modem, then to my router. Then to a switch and a line from the switch goes to my Proxmox server via it's ethernet connection on the motherboard. So far so good. I create the Proxmox server and I can attach to it via it's webpage. All good. NOW, in my Proxmox server I have a 4 port ethernet PCIe card (Intel). Nothing is attached at this point. I next go to the Proxmox webpage on my server and install pfSense. Once it's installed, I am prompted to insert the WAN connection.... and here's the problem.... if I unhook the WAN line from my Internet suppled router and plug it into the Intel 4-port card of the Proxmox server, I loose connection to my Proxmox server, which keeps me from configuring Proxmox or the pfSense VM. I was able to get pfSense to work but I have to keep switching connections back to the motherboard ethernet connection in order to maintain the VM. Is there any way to so me a drawing or explain how to access the Proxmox server once connected to pfSense?? My understanding is that pfSense is supposed to replace my ISP supplied router. Am I misunderstanding something here? Thank you for any help you can provide.
When trying to install opensense and pfsense I got the error "IOMMU not enabled". I checked in the BIOS that Intel vt-d was enabled and then found out by googling that I had to edit the Proxmox /etc/default/grub file to include GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on" then update-grub and reboot The vm still cannot start. Now I get "QEMU exited with code 1" ***Update*** I remembered that this series 2 I5 computer has problems running Qubes so I switched to series 3 I5 computer (both Fujitsu Esprimo) and now Opnsense works fine. I have an Intel 4 port NIC and had to switch off both "all functions" and "rombar" to get it to work. PFsense crashes with a kernel panic in the VM but I prefer Opnsense anyway. ***Update*** I discovered more of your great tutorials and edited /etc/modules to add the lines: vfio vfio_iommu_type1 vfio_pci vfio_virqfd Now pfsense works too ! Thank you so much Tim !
Hi TechnoTim, this was a great tutorial. I followed it almost successfully, all my LAN client are getting IP addresses except for the guest VMs that rely on the vmbr NIC. Did you come across this and if so how did you resolve it? Many thanks
This video was awesome. While we are on the subject of virtualizing firewall: Can you add a third NIC to the PFsense VM that is also on the LAN side but its inside the Proxmox virtual environment? What I mean is, for physical devices on the LAN side you would connect it to the LAN physical port (maybe add a switch first), but for the other VMs that live on the same Proxmox host as the Pfsense, it would be a waste to send their traffic out a phisical port then back on the LAN port. Is my assumption correct that all you would have to do is create a new linux bridge in proxmox (vmbr2 maybe) and just add that as a third adapter to pfsense and configure it as LAN. Then from there just add that bridge as an adapter to all your VMs?
any updates on this? I am doing this ahem... 2 years later - but yes - doing it.. I haev OPNSense in a VM - but instead of passing the physical pci device why not virtualize that so we can make use of HA services on Proxmox ? Make the failover mirror on proxmox on another server with similar hardware setup - then if one machine needs to be shutdown for updates or maintenance - easy to migrate to other node on the cluster and keep everything going... no? Would love to see how this is done in your world.
Great tutorial. I really like how well you laid out this content. I'm a network engineer and while I knew how to do all of this networking, I wanted to see how you explained it for laymen. Fantastic stuff. I also completely muffed my own proxmox setup, I didn't realize you could pass through NIC's so easily. I made an OVS bridge for the WAN, I don't want to talk about it :( One little change I would make is on the LAN gateway address. While you can always make the gateway whatever IP you want on the subnet, I really like to keep it to either the first address in the subnet, or the last address in the subnet. Remembering a random address is difficult years down the line and if you ever need to add a statically configured network device, its easier to remember first address or last address. Anyway, just my $0.02.
Is it possible to utilize pfsense on proxmox using only laptop with one NIC using VLANs. I know you elaborated on these subjects but not in such combination. Thanks for you help
IOMMU dont forget that. But how do i enable it? I have tried before but idk if i got it work in bios i have vtd and trusted and Intel Virtualization Technology for Directed I/O but trusted is off
Even though I have a PCI network card with two ports, adding them as PCI cards in Proxmox did not work for but instead as NICs, the rest was flawless, thanks for the video man, I dropped a sub as well.
I'd do this, but there is a major downside. When you have to take your ProxMox server it's hosted on down, or it goes down, there goes the internet down for everybody. You might need that internet to download something to get your server back up... which means you'll be digging out your dedicated DD WRT router appiance and plugging it all back in again. I'd keep it at the handy to do the switcherroo in a pinch. My appliance is a WRT3200ACM which is more than beefy enough for anything I throw at it, over kill even... so no benefit for me to do this.
question how can include proxmox web on same network as you pass hardware pci direct to pfsense im trying to acess proxmox direct from pfsense network ?
Perhaps good thing to mention in a comment is that you need IOMMU enabled. I went and watched your "before I do anything" video and you explained it great there. Quick reference would be nice because I got stuck when I wanted to start the VM.
@Régis Loyauté The fact I didnt know kind of hightlights the absence of common knowledgde. These videos arent made for veterans of virtualisation as far as I'm aware.
This something worth noting indeed. I personally ran into lack of IOMMU on one older hardware. Let's be real. There's a lot of vids that recommend turning old device into Proxmox server and in certain situations user will severly get hampered with lack of its support. I was looking into sharing gpu to vm and ran into lack of IOMMU hardware support.
please make video about openmediavault with proxmox, how the right way we do the config, for share storage, and storage for CCTV using FTP/SFTP protocol and others what that openmediavault can do, by the way, thanks for explaining clearly, i like a way you explain
Thank you for this video, and for your channel. I do have a question. I have a similar setup as seen in the 2:22 mark of this video (onboard NIC and dual NIC card). My onboard NIC is attached to my switch via a green cable. My WAN port is plugged into my provider's cable modem via a white cable and my LAN port is plugged into my switch via a black cable (BTW, same switch that the onboard NIC is plugged into so that I can go to Proxmox web UI). pfsense seems to be working with this setup, but how do my Proxmox VMs get their Internet? Since the dual NIC card is being passed through to the pfsense VM, and other VM will not see this card. Is there something I need to do in Proxmox or pfsense to bridge the two?
just fantastic. I have been prepping my own home server and was sweating because I wasn't sure what to do to isolate it from the network. "Is it safe to host?" "whats pfsense even do" "should i buy dedicated hardware" "where WAS that lasagna!?!" and this video made it so clear. Thank You
Just wanted to let you know that the German subtitles on this video are absolute garbage. They look like a machine translation plus Google Translate had a bad day. I recommend removing them as they aren't even readable half of the time.
Hi Techno Tim, I’m new to proxmox, and I’ve been trying to follow this. I have two 10gb nics which i am passing both via pci as per the video, however as soon as I start the VM I loose network connection to the device. It seems since I’m passing the hardware, the ssh/proxmox’ webinterface become unavailable. What should I be doing differently? thanks again.
On a server with only 2 NICs, instead of passing both directly to pfsense via PCI passthrough, is it possible for pfsense to see only 1 physical NIC (via passthrough) and also 1 virtual NIC, where the virtual NIC is configured to bridge Proxmox with the other physical NIC? I think that would solve the problem, and it's what I hope to do as well.
@@yeahforbes also adds security issues. really should dedicate 2 nics to the vm. and have 1 dedicated to proxmox. so really need 3 physical ports on your machine for basic setup
@@bandit8623 I implemented my 2 nic idea soon after I wrote about it, and it's been working for me since then. What is the threat model you're referring to?
@@yeahforbes if you have a nic not direct pathed to the vm it has more chance for vulnerabilities. especially if you have other vms on that same shared nic.
Is this possible to do with only 2 ethernet ports? I have a pcie card with 1 ethernet port, and I also have the standard one on the motherboard. In 2:22 I can see that the red wire is probably connected to whatever computer is used to connect to the proxmox web interface. Trying it out for myself with just 2 ports made my setup, as expected, go down :) I will try again with a USB-ethernet dongle or the onboard wifi (if I can get it to work) so I can access the web-interface..
for those who cannot buy dual nic please make a tutorial for vitualizing nics too (if possible) There is virtual switch possibility on proxmox but have no idea how to set safely. please advise.
I ran into the error:kvm: -device vfio-pci,host=0000:03:00.0,id=hostpci1.0,bus=pci.0,addr=0x11.0,multifunction=on: vfio 0000:03:00.0: device is already attached TASK ERROR: start failed: QEMU exited with code 1 this was because I followed exactly what you did and ticked the box 'enable all functions' on all four ethernet ports of my pcie quad nic card. This made it so it tried to bind the whole card 4 times which isn't possible. Not ticking the box for all four of them solved the issue for me.
Warning: if something goes wrong with your virtualisation platform, you lose internet access, unless you have a multi-node cluster. In line with enterprise convention, I tend to keep critical things (which usually change rarely) separate from non-critical things (which tend to change more frequently). My NAS/virtualisation host changes far more frequently than my firewall, and I want my firewall to be up, even if my NAS is down - in fact, I need my firewall up _especially_ when my virtualisation host is down.
I get an error when the VM starts up: vfio-pci 0000:03:00.0. Invalid PCI ROM header signature: expecting 0xaa55, got 0xffff Since it's configured to run on startup my whole server is now down.
I went through the same research journey around the same time. I also seriously thought about putting pfSense on virtual machine. Eventually I decided to purchase a dedicated hardware for pfSense because of all the reasons people talked about on the internet. I probably would try to visualize it if I saw your video earlier. Now my whole set up is already completed, and it's very stable. I don't want to mess with it.
This may be a stupid question, but I'm trying to install pfSense on a VM on my Proxmox server where I just installed a new dual Ethernet port card. However, it's currently plugged in via the Ethernet port that came with the server. It sounds like in order to complete the pfSense install, both WAN and LAN ports must be plugged in and link must be up. Should I plug one of the ports on the new card to the WAN port on the carrier's modem and the other port on the new card on the Ethernet switch I have? If I do this, I would lose internet access momentarily, until the installation and configuration is finished. I guess I would need to access the shell directly on the server, not the web GUI. Also my Proxmox currently has vmbr0 Linux Bridge set up. I suppose I would keep that for the VMs I've been running on it but for the pfSense VM should I create a new Linux Bridge? Thanks in advance.
two things....why did you add pci device and not network device card as i've seen in all other similar vids?....secondly, as feedback - thanks for posting. apart from knowledgeable and simple to follow, it's calm and easy to listen to...
Need your help. looks like you have already fixed the PCI pass through issues beforehand so you are not discussing in the video here. As such my question and your answer will help lot of people who will come here. I am on 6..24 proxmox and done all updates yesterday. I have HP NC364T PCI-E Quad Port Gigabit Ethernet adapter IBM 39Y6137 Intel PRO/1000 installed before I build proxmox. I read that this Card is working with Proxmox. When in the node->Hardware->Network I can see 5 NICs (one built in to Dell and 4 from the Pcie card). When I try to add PCI device for PFsense VM as you did in your video I get "No IOMMU detected, please activate it.See Documentation for further information". displayed on the top which was not in your video. So my question is a) is there any stetting in hardware bios? b) Do I have to follow this guide pve.proxmox.com/wiki/Pci_passthrough and do something c) Do I have to create linux bridges for additional NICs? One person had the following method to solve similar issue but not sure as my one if not specific to NICs but for all PCI devices create the directory /etc/default/grub.d create a file ending with “.cfg” in that directory. add the following to it: Code: GRUB_CMDLINE_LINUX_DEFAULT="${GRUB_CMDLINE_LINUX_DEFAULT} intel_iommu=on" run update-grub. Rebooted and everything works- see here Kindly advise what I need to do. Thanks
I had the same issue with IOMMU, guessing because I am using a Core i7 and not a Xeon processor. I just followed instructions on pve.proxmox.com/wiki/Pci_passthrough. Solved that problem . Had the issue @vfxfan had as well, so just followed instructions as listed. Guessing when I do PCI passthrough it is passing through the entire card, therefor passing through all the NICs together :)
Might be late to the party, followed your video and worked perfectly (thank you) only thing is if I reboot the vm (for pfsense) I don't get a WAN ip back, only way to get it is to reboot the Proxmox server, can't find anything to point me to the correct direction
is there a way to make segmentized network inside of virtualized firewall? i mean to deliver tagged vlans to pfsense or in my case Sophos XG Home firewall, through truenas (in my case Scale) thanks :)
How does this work when you want to use this as your main router from your ISP. Currently I have a consumer router that allows me to connect to proxmox. How do I swap over to pfsense by plugging my wan port into proxmox and having my lan still work. Do I need to configure proxmox in the same address space as my current router?
Hi Nice Vid!! At 3.48 you mention that you can pass-through of a 4 nic card only the 1/4 portion of it?? How is that possbile? I am used to Unraid on which you need to exclude the specific pci device you want to pass first and afterwards to give it to the VM. Even more difficult if that device is a motherboard controller (usb, nic). Is it possible in Proxmox to pass-through motherboard controllers without braking things? Isn t in Proxmox mandatory the passed through device to be in its own iommu (so iommu capable motherboard needed?) Last but not least did you have to put your isp's modem in bridged mode in order for this to work? Thank you
ierosgr you can split up IOMMU groups in Proxmox with config! Not sure about splitting integrated motherboard items. Didn’t have to do anything different for my ISP’s modem. A NIC is a NIC to them.
@@TechnoTim At the start of the video you show up an intel nic which is an external pci device. Afterwards show you passing through a broadcom (so an intergraded one) Which of the two did you pass to Pfsense? Why a nic card differ from a gpu device and you dont have to enter conf files to exclude it from Hypervisor at boot like a gpu?
Hi Tim. You need to put a space before 'Techno' for the link to the HP Dual Gigabit NIC so the link works.
3 года назад
Tnks for the help, @Techni Tim! If anyone get a error like this -> "TASK ERROR: KVM virtualisation configured, but not available. Either disable in VM configuration or enable in BIOS." - Please, follow this steps to solve! Bye!
Hi Techno Tim. I've tried to setup pfSense as you di but everytime i start the pfSense VM i've Lost access to proxmox even if i have the server connect with a cable to LAN access with the ISP modem/router and a cable of my PCIE card connected with the WAN of the ISP. I've tried also to check the Netscape guide to virtualize pfSense on Proxmox but in that case pfSense can't take any WAN IP. Any suggestion on what to do?
Just did this but having issues connecting to the internet. WAN dhcp is enabled and pulls an IP but nada. Any ideas? I did have IOMMU issues went through all that and got that resolved. I have a four port pci e nic that is being used for my wan and lan. I can access the gui but no internet.
I did not virtualize my router because the router has to be dedicated to it's own machine and I repurposed my Nuc because it has two Intel Nics and this machine is my slowest computer. Go Intel!
if i get this right, the one that goes out is goin into a switch, than its going back into my server (motherboard) lan, so my other vms have internet access as well.
Great Video - first TechnoTim I have seen. Great job explaining and sharing. I have been using pfSense about 2 years now on an HP t620+ ThinClient with an added 2-port Intel i350-T2 card. Been working great, but I have this awesome Workstation class machine I want to use for ProxMox. I have 8.0.9 installed there, and I am just beginning. I purchased a 4-port i350-T4V2 for this box, and it is working fine. In the t620+ I had disabled the on-board NIC as was not using it. I know that ProxMox requires a NIC for accessing the host/dashboard, but can it be one of the 2-ports I will use on the i350-T4? I have a cable from Cable modem to port 0 on the 4-port and cable from port 1 to the Netgear Orbi (wifi AP)...as it has a satellite in the other end of the house where the office is - so that I have Wired (per se) access back there and wifi is stronger. From the Orbi (at the ProxMox box & modem - there is a cable into the on-board NIC of the ProxMox host). If I unplug this, I lose access to the host dashboard.
Hi Tim, great tut. Had to do some IOMMU separation to get it to work but finally did it and working. Now, I have PFsense running inside vm giving its own network and dhcp to everything comming out through the lan port. So far so good. I want now to place the proxmox host behind pfsense as well and leave the primary modem only passing traffic to pfsense with DMZ. I just need to plug the nic (using proxmox) to the switch but before change de ip address? I'm not sure how to do this.
If Pfsense is running within Proxmox and connect to a modem, isn't promox exposed to the internet. Is it safe to just reroute all the traffic go through Pfsense including the other VM and CT?
I have a error if I try to start pfsense in proxmox. Is there a way to solve this. :( kvm: -device vfio-pci,host=0000:08:00.0,id=hostpci1.0,bus=pci.0,addr=0x11.0,multifunction=on: vfio 0000:08:00.0: device is already attached TASK ERROR: start failed: QEMU exited with code 1
I'm sure it's been covered (in fact I know of 1 other creator that has) but running Unraid on Proxmox, I followed his skim-through and I can see it in the console but cant connect. Maybe in it elaborate on selecting network interfaces (cards) to split them among the chassis (Proxmox) and vms (PfSense, Unraid, and TrueNAS at least) And longshot but if you have a multi-day chassis (like my sc846) how to specify specific bays to certain vms (not specific drives, that way any drive inserted into "bay 20" will be assigned to vm X.
@@TechnoTim if u blacklist your ethernet cards like with GPU does that mean other VMs don't get internet? this process doesnt seem straightforward and i cant find alot of resources online for NICs
My concern would be latency and in particular erratic latency. My router and gateway are dedicated purpose built hardware. Clearly I'm not undermining your video or intention, just a side note. I play twitch (FPS) games where a stable low latency is king. Great video 👌
Thanks for the comment! My pings are pretty low and consistently low (26 ms to Overwatch servers, 40 ms for Apex).All connections are physical since I am passing through the hardware to the VM and everything else is in memory.
@@TechnoTim .. It's all the background tasked to the CPU in a VM, I'm not even remotely suggesting it can't or shouldn't be done. Maybe a comparison video 😎👍 Thanks for the reply and good luck on your channel, interesting videos.
@@DJaquithFL I could answer some of this. Most routers will use very lower power and small amounts of RAM, you would in most commercial devices have a 400Mhz CPU and around 128MB Ram. therefore the foot print of this is small on a "VM" providing that the node is not over loaded, modern and typically not slow or performance degraded via other software running on the same hardware.
@@guywhoknows .. Just my observations .. I have the best experienced when I'm at home alone not sharing our 500/500 Mbps connection even with a QoS. FPS gaming is about milliseconds (ms), priority and not losing packets in UDP. Example of 2 people shoot simultaneously: 1. Lost Packets are the same as you not shooting; 2. Latency spikes up even a few milliseconds the server reads as your death, first shot wins. Now if you're streaming e.g. Netflix via TCP .. latency spikes, lost packets, buffer bloat, etc are not noticeable and therefore irrelevant.
Hi TechnoTim, I hope you are able to answer one silly question about this setup: When experimenting with different virtualised router OSes I find the default LAN networks vary from product to product. And I like to just use the defaults most of the time in case changing them gives unexpected problems. This gives me a quandary about where to put my PVE management interface. I prefer to put it on the LAN, but that means it invariably ends up on a network number different from whatever I'm running for a router. So I have no access unless I mess with my network settings on my PC. Then I have to change them back to test out the router behaviour. I just wondered how you manage this problem in your setup, or do you just live with it?
I've tried this guide twice now, and it bricks my Proxmox file system every time forcing fresh install. I think it's either the VirtIO Block setting, or adding the PCIE card directly. Could you explain those more?
I was considering virtualising pfsense in proxmox but concerned about exposing my proxmox machine directly to the internet by placing it upstream of the firewall it is virtualising. Is this a concern? How did you mitigate this please. Thanks
Hi, Do you know how I could enable my computers that obtains network access from the virtual pfsense to be able to reach the proxmox servers webinterface? So for example if I have a virtual PC hosted on proxmox and its network is obtained from the virtual pfsense, how would I then allow for that virtual PC to be able to reach the proxmox interface?
@Techno Tim Thank you for the great video! I'm just scoping out the work I have a head of me, and want to know, can you access the proxmox UI via web from an IP dealt by the pfsense VM? Ideally i would like proxmox to be accessible from the virtual router, instead of the physically accessing the proxmox service with a keyboard and mouse. So my usecase is simple: access proxmox from my desktop that is connected to my virtual pfsense router.
Very helpful video, thanks! I have a question though if you don’t mind! Say i create a linux bridge to the passed-trough LAN port to allow connectivity between my other VMs and the physical switch managed by pfsense. Will the VMs bypass the pfsense firewall? Or will they be routed trough it? Thanks!
For starters this router Software going to cause some headache, especially for those must log into the isp via pppoe. In that case, if you have no ip on wan during first configuration of the interfaces, doesnt matter, you have to log into the web configurator and configure your wan interface, if still no change is detected, swap your cables and done, but with only two interfaces, if you cant get into the web ui, you have to swap cables and done, configure your wan. After all, i saw a mistake in the video, he accidentally or not, passed the same pci device to the vm twice, secondly you not really explained why you choosed the gateway to be on the ip 11. So this can be a little confusing.
what budget hardware do you recommend i buy so i can build a hardware pfsense to use as a firewall. i don't want to virtualize it. gonna use it at home non commercial use.
So if your server is virtualizing your router, how would you connect the server to the internet? Like, how would the other VMs on the server have access to the internet? Would you have to plug the ethernet port on your motherboard into the switch that your NIC is plugged into?
@@TechnoTim Hi Tim, thanks for answering my question on such an old video! I'm in the middle of setting up my first virtualization server and your videos have been an invaluable resource. You've definitely earned a subscriber for life! 🙌🙌🙌
i now can run all my pfsnese on one server having over 10 ips and wanting firewall protection for all was a headache. now with all of them on one machine i can monitor them easier than before. Just a note if running nic that have 4 ports i didn't check the all function it would disable the 4 port to a 2 port for some weird reason.
Hello Tim. I'm banging my head around a setup with a Chinese 6x ports firewall, proxmox and a openwrt lxc. I can't see LuCi, and the structure of CIDR and gateway is obscure to me. Is there a forum I can meet with you and have a chat? Are you on the proxmox forum?
Which firewall / router are you running at home? If you can't remember, maybe it's time to SWITCH ;)
By the way, if you're new here, welcome! Please remember to ✨subscribe✨ for more content like this!
Used Pfsense since 3 years back inside of a proxmox machine at home. Coupled together are a XCP-NG machine running Pfsense at my parents house with IPsec tunneling. (150km) Getting 250/250mbit sym bandwith between the two places. :)
Nice! That's awesome you have a hypervisor at your parents house! Why did you choose XCP-NG over proxmox?
@@TechnoTim I think both are realy good products. XCP-ng do have a easier time to bind a swarm of servers in my opinion. And i do like the disaster recovery, XenMotion, True backup system compare to tar.gz of proxmox. But as i said. You can't go wrong with either.
just a question, but is it safe to use qnap nas virtual station for pfsense vm? compared to using say a nuc install ubuntu? then install proxmox? then install pfsense freebsd as a vm using that proxmox?
no portforwarding being done, just merely using as an edge router firewall.
XD
Always log on with the new account before disabling the old account.
Lmao risky
I have an identical setup. One thing to consider depending on how many cores you have on the host, is to make the CPU type 'host' and pass through 1 or 2 physical cores. This should ( depending on your CPU ) enable the AES-NI CPU crypto which can be useful if you use OpenVPN and want faster throughput over encrypted connections. Awesome guides by the way, I wish these vids were around years ago!
Good call!
Incredible quality, easy to understand, as always fantastic! Thanks for your videos Tim, keep doing them please.
You got it! Thank you!
Heyo Tim, you have greatly helped me get into the Homelab scene, and I appreciate it. With that said, you really should consider revisiting this video with a 2022/2023 edition. Reason why I say this is because passing my NIC down to the OPNSense VM in Proxmox (and even Pfsense) straight up did not work. I almost gave up, until I talked to someone that had a workaround: by creating a Linux bridge with the NIC as an alternative way. Passing the NICs down did not work but creating a bridge did. I had other people express their grievance about following your video and having it not work. And from what I heard, when it comes to virtualizing routers/firewalls, passing down NICs is a huge NoNo for this reason. I have no doubt this worked for some people, but I feel like there is a higher chance of success with an updated video by using the create Linux bridge method. Just my 2 cents!
He recommended the first method likely because it maximizes performance to just pass through, but bridging through a virtual interface works likely as well. His suggestion is ideal when supported. If not, with VM's you can typically emulate a method instead.
Pass through in general can get finicky for some.
I had issues trying to passthrough my 4port NIC card. I ended up unchecking the "all functions box" and that solved my problems.
Such a great idea for those tech heads that want to do something more than what those basic modem routers.. Just a note for those with different NBN connections that you may still need the netgear/gateway/modem from your ISP but simply put it into bridge mode then pass that to the WAN interface as per TechnoTim's guide!! (suit most Australian NBN type of setups) As I am and Aussie viewer also!!!
don't forget to enable IOMMU. The version of Proxmox 6.1-7 didn't enable it by default.
Good call! Sorry, I already had it enabled from a previous video >.
@@TechnoTim yea, took me a good hour to figure get my R410 working correctly with IOMMU.
Thank you for this video. I have one “noob”question. Using a physical machine that has 6 network ports, running ProxMox and a pfSense VM...how can I access ProxMox web control panel from my network that is being served by pfSense? Do I just need to ensure ProxMox is on the same subnet as my LAN? Thank you kindly for helping.
I am wondering how you interact with Proxmox after you virtualize your network as a VM through Proxmox... I am wondering how the system determines an IP through a VM that hasn't booted yet. After it boots, how does it get an IP from the VM?
Maybe you should do a video on setting up Vlans on proxmox?
Thank you!
yes in deepth review and tutorial is much needed. i hope he would do it
If your CPU supports AES-NI and you like to use it in your pfSense/OPNsense VM for OpenVPN etc. you can change processor type to "host"
Good call!
Basically you always want to use host, unless you want to do a live migration to a different host with a different cpu. Kvm has the bare minimum of cpu flags, host type is always better
@@tomashrubovcak3770 Hm, yeah, sounds reasonable. Any idea why proxmox defaults to KVM ?
@@succubiuseisspin3707 precisely for live vm migration reasons. I learned that the hard way when I couldn't figure out why my tls offloading proxy was so slow on my proxmox vm... Then I dug around and found some official docs covering that.
Any chance you could do a video on how to passthrough hard disks to a VM in Proxmox for FreeNAS virtualization?
If I can somehow acquire more hardware I'd love to!
Nfs share will do the job
Hi Tim fantastic video!
I'm just getting started with Proxmox but so far I am digging it, I want to set up a virtual PFsense instance but not to act as my real firewall in my office, I just want to be able to join other VM’s within Proxmox to the LAN network that PFsense is creating.
That way I could test VPN solutions like Wireguard, Zerotier and Open VPN from one VM to another that are on different networks.
My Proxmox box does have 2 NICS, actually 3, what would be the best way to go about this?
I feel like I can basically follow your tutorial except for on the LAN NIC for PF sense I don't need to connect it to a switch I just need it to broadcast to the other VMS in Proxmox, just not quite sure how to do that.
Thanks !
Can I ask a question, probably a stupid one but here goes... How are the two wired? I have my internet coming into my home to my modem, then to my router. Then to a switch and a line from the switch goes to my Proxmox server via it's ethernet connection on the motherboard. So far so good. I create the Proxmox server and I can attach to it via it's webpage. All good. NOW, in my Proxmox server I have a 4 port ethernet PCIe card (Intel). Nothing is attached at this point. I next go to the Proxmox webpage on my server and install pfSense. Once it's installed, I am prompted to insert the WAN connection.... and here's the problem.... if I unhook the WAN line from my Internet suppled router and plug it into the Intel 4-port card of the Proxmox server, I loose connection to my Proxmox server, which keeps me from configuring Proxmox or the pfSense VM. I was able to get pfSense to work but I have to keep switching connections back to the motherboard ethernet connection in order to maintain the VM. Is there any way to so me a drawing or explain how to access the Proxmox server once connected to pfSense?? My understanding is that pfSense is supposed to replace my ISP supplied router. Am I misunderstanding something here? Thank you for any help you can provide.
When trying to install opensense and pfsense I got the error "IOMMU not enabled". I checked in the BIOS that Intel vt-d was enabled and then found out by googling that I had to edit the Proxmox /etc/default/grub file to include
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
then
update-grub and reboot
The vm still cannot start. Now I get "QEMU exited with code 1"
***Update***
I remembered that this series 2 I5 computer has problems running Qubes so I switched to series 3 I5 computer (both Fujitsu Esprimo) and now Opnsense works fine. I have an Intel 4 port NIC and had to switch off both "all functions" and "rombar" to get it to work. PFsense crashes with a kernel panic in the VM but I prefer Opnsense anyway.
***Update***
I discovered more of your great tutorials and edited /etc/modules
to add the lines:
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
Now pfsense works too ! Thank you so much Tim !
Can you just run me through how you would setup a DMZ that contains some vms?
Give your pfsense server a new virtual nic. Then create a new virtual network. Block all traffic from the newly create VM. The rest is up to you!
Hi TechnoTim, this was a great tutorial. I followed it almost successfully, all my LAN client are getting IP addresses except for the guest VMs that rely on the vmbr NIC. Did you come across this and if so how did you resolve it? Many thanks
@guya4007
Did you ever solve your issue. I have Exactly the same issue
Hey Tim, is the performance going to be the same or better if I have symmetrical gigabit connection using 3700x virtualization?
The same
@@TechnoTim thanks
This video was awesome. While we are on the subject of virtualizing firewall: Can you add a third NIC to the PFsense VM that is also on the LAN side but its inside the Proxmox virtual environment? What I mean is, for physical devices on the LAN side you would connect it to the LAN physical port (maybe add a switch first), but for the other VMs that live on the same Proxmox host as the Pfsense, it would be a waste to send their traffic out a phisical port then back on the LAN port. Is my assumption correct that all you would have to do is create a new linux bridge in proxmox (vmbr2 maybe) and just add that as a third adapter to pfsense and configure it as LAN. Then from there just add that bridge as an adapter to all your VMs?
Great stuff!!
Any chance you could do a video on how to create an AP too using the integrated wifi adapter many repurposed homelab computers have? :)
1.proxmox can do hardware accelaration from pfsense through nic ?
2. there is option to define standard vSwitch in proxmox like vsphere ?
any updates on this? I am doing this ahem... 2 years later - but yes - doing it.. I haev OPNSense in a VM - but instead of passing the physical pci device why not virtualize that so we can make use of HA services on Proxmox ? Make the failover mirror on proxmox on another server with similar hardware setup - then if one machine needs to be shutdown for updates or maintenance - easy to migrate to other node on the cluster and keep everything going... no? Would love to see how this is done in your world.
Tim, your videos are invaluable. Thanks for the amazing work, you TRULY deserve like 1 MLN subscribers already.
Great tutorial. I really like how well you laid out this content. I'm a network engineer and while I knew how to do all of this networking, I wanted to see how you explained it for laymen. Fantastic stuff. I also completely muffed my own proxmox setup, I didn't realize you could pass through NIC's so easily. I made an OVS bridge for the WAN, I don't want to talk about it :( One little change I would make is on the LAN gateway address. While you can always make the gateway whatever IP you want on the subnet, I really like to keep it to either the first address in the subnet, or the last address in the subnet. Remembering a random address is difficult years down the line and if you ever need to add a statically configured network device, its easier to remember first address or last address. Anyway, just my $0.02.
Thank you so much!
Makes a lot of sense
Is it possible to utilize pfsense on proxmox using only laptop with one NIC using VLANs. I know you elaborated on these subjects but not in such combination. Thanks for you help
IOMMU dont forget that. But how do i enable it? I have tried before but idk if i got it work in bios i have vtd and trusted and Intel Virtualization Technology for Directed I/O but trusted is off
Thank you!
What's your motherboard?
@@stuffinfinland idk, it's in the dell t1600
How does this work with your ubiquity gear (udm-pro)? I’m in a similar situation and just wanted your thoughts.
Even though I have a PCI network card with two ports, adding them as PCI cards in Proxmox did not work for but instead as NICs, the rest was flawless, thanks for the video man, I dropped a sub as well.
If I disconect the ISP router to connect to the WAN port, then who is providing the IP address to proxmox so I can control it?
I'd do this, but there is a major downside.
When you have to take your ProxMox server it's hosted on down, or it goes down, there goes the internet down for everybody.
You might need that internet to download something to get your server back up... which means you'll be digging out your dedicated DD WRT router appiance and plugging it all back in again. I'd keep it at the handy to do the switcherroo in a pinch.
My appliance is a WRT3200ACM which is more than beefy enough for anything I throw at it, over kill even... so no benefit for me to do this.
Seriously the most helpful tutorials on RUclips, thank you!
Glad it was helpful!
question how can include proxmox web on same network as you pass hardware pci direct to pfsense im trying to acess proxmox direct from pfsense network ?
Perhaps good thing to mention in a comment is that you need IOMMU enabled. I went and watched your "before I do anything" video and you explained it great there. Quick reference would be nice because I got stuck when I wanted to start the VM.
@Régis Loyauté The fact I didnt know kind of hightlights the absence of common knowledgde. These videos arent made for veterans of virtualisation as far as I'm aware.
This something worth noting indeed. I personally ran into lack of IOMMU on one older hardware. Let's be real. There's a lot of vids that recommend turning old device into Proxmox server and in certain situations user will severly get hampered with lack of its support. I was looking into sharing gpu to vm and ran into lack of IOMMU hardware support.
If my server already has a 4 port NIC, and I'm only using one port. Would I still need to install a seperate PCI NIC?
please make video about openmediavault with proxmox, how the right way we do the config,
for share storage, and storage for CCTV using FTP/SFTP protocol and others what that openmediavault can do,
by the way, thanks for explaining clearly, i like a way you explain
Hey sorry. I don’t use omv
Thank you for this video, and for your channel. I do have a question. I have a similar setup as seen in the 2:22 mark of this video (onboard NIC and dual NIC card). My onboard NIC is attached to my switch via a green cable. My WAN port is plugged into my provider's cable modem via a white cable and my LAN port is plugged into my switch via a black cable (BTW, same switch that the onboard NIC is plugged into so that I can go to Proxmox web UI). pfsense seems to be working with this setup, but how do my Proxmox VMs get their Internet? Since the dual NIC card is being passed through to the pfsense VM, and other VM will not see this card. Is there something I need to do in Proxmox or pfsense to bridge the two?
Make sure you create a network bridge where all your physical and virtual devices can communicate with each other
just fantastic. I have been prepping my own home server and was sweating because I wasn't sure what to do to isolate it from the network.
"Is it safe to host?"
"whats pfsense even do"
"should i buy dedicated hardware"
"where WAS that lasagna!?!"
and this video made it so clear. Thank You
Just wanted to let you know that the German subtitles on this video are absolute garbage. They look like a machine translation plus Google Translate had a bad day. I recommend removing them as they aren't even readable half of the time.
Thank you! I relied on the "machine" . I will just have to forgo them for now. Thank you for the feedback!
Could I use the motherboard NIC as WAN and the PCIE NIC as LAN? Made the mistake of buying a single port NIC
Hi Techno Tim, I’m new to proxmox, and I’ve been trying to follow this. I have two 10gb nics which i am passing both via pci as per the video, however as soon as I start the VM I loose network connection to the device. It seems since I’m passing the hardware, the ssh/proxmox’ webinterface become unavailable. What should I be doing differently? thanks again.
Do you have a 3rd NIC?
On a server with only 2 NICs, instead of passing both directly to pfsense via PCI passthrough, is it possible for pfsense to see only 1 physical NIC (via passthrough) and also 1 virtual NIC, where the virtual NIC is configured to bridge Proxmox with the other physical NIC? I think that would solve the problem, and it's what I hope to do as well.
@@yeahforbes also adds security issues. really should dedicate 2 nics to the vm. and have 1 dedicated to proxmox. so really need 3 physical ports on your machine for basic setup
@@bandit8623 I implemented my 2 nic idea soon after I wrote about it, and it's been working for me since then. What is the threat model you're referring to?
@@yeahforbes if you have a nic not direct pathed to the vm it has more chance for vulnerabilities. especially if you have other vms on that same shared nic.
Is this possible to do with only 2 ethernet ports? I have a pcie card with 1 ethernet port, and I also have the standard one on the motherboard. In 2:22 I can see that the red wire is probably connected to whatever computer is used to connect to the proxmox web interface.
Trying it out for myself with just 2 ports made my setup, as expected, go down :)
I will try again with a USB-ethernet dongle or the onboard wifi (if I can get it to work) so I can access the web-interface..
for those who cannot buy dual nic please make a tutorial for vitualizing nics too (if possible) There is virtual switch possibility on proxmox but have no idea how to set safely. please advise.
You can also do it with one additional, just create a linux bridge!
Proxmox running PFSense + TruNAS Scale + Ubuntu ++ PlexServer ++ SMBA ++ Home Security Video/Audio/Alarm
I ran into the error:kvm: -device vfio-pci,host=0000:03:00.0,id=hostpci1.0,bus=pci.0,addr=0x11.0,multifunction=on: vfio 0000:03:00.0: device is already attached
TASK ERROR: start failed: QEMU exited with code 1
this was because I followed exactly what you did and ticked the box 'enable all functions' on all four ethernet ports of my pcie quad nic card. This made it so it tried to bind the whole card 4 times which isn't possible. Not ticking the box for all four of them solved the issue for me.
Warning: if something goes wrong with your virtualisation platform, you lose internet access, unless you have a multi-node cluster.
In line with enterprise convention, I tend to keep critical things (which usually change rarely) separate from non-critical things (which tend to change more frequently). My NAS/virtualisation host changes far more frequently than my firewall, and I want my firewall to be up, even if my NAS is down - in fact, I need my firewall up _especially_ when my virtualisation host is down.
I get an error when the VM starts up:
vfio-pci 0000:03:00.0. Invalid PCI ROM header signature: expecting 0xaa55, got 0xffff
Since it's configured to run on startup my whole server is now down.
I went through the same research journey around the same time. I also seriously thought about putting pfSense on virtual machine. Eventually I decided to purchase a dedicated hardware for pfSense because of all the reasons people talked about on the internet. I probably would try to visualize it if I saw your video earlier. Now my whole set up is already completed, and it's very stable. I don't want to mess with it.
Whatever works for you!
This may be a stupid question, but I'm trying to install pfSense on a VM on my Proxmox server where I just installed a new dual Ethernet port card. However, it's currently plugged in via the Ethernet port that came with the server. It sounds like in order to complete the pfSense install, both WAN and LAN ports must be plugged in and link must be up. Should I plug one of the ports on the new card to the WAN port on the carrier's modem and the other port on the new card on the Ethernet switch I have? If I do this, I would lose internet access momentarily, until the installation and configuration is finished. I guess I would need to access the shell directly on the server, not the web GUI. Also my Proxmox currently has vmbr0 Linux Bridge set up. I suppose I would keep that for the VMs I've been running on it but for the pfSense VM should I create a new Linux Bridge? Thanks in advance.
If our server has multiple nics can we just use one we’re not currently using instead of installing a separate card can’t we?
two things....why did you add pci device and not network device card as i've seen in all other similar vids?....secondly, as feedback - thanks for posting. apart from knowledgeable and simple to follow, it's calm and easy to listen to...
Need your help. looks like you have already fixed the PCI pass through issues beforehand so you are not discussing in the video here. As such my question and your answer will help lot of people who will come here. I am on 6..24 proxmox and done all updates yesterday. I have HP NC364T PCI-E Quad Port Gigabit Ethernet adapter IBM 39Y6137 Intel PRO/1000 installed before I build proxmox. I read that this Card is working with Proxmox. When in the node->Hardware->Network I can see 5 NICs (one built in to Dell and 4 from the Pcie card).
When I try to add PCI device for PFsense VM as you did in your video I get "No IOMMU detected, please activate it.See Documentation for further information". displayed on the top which was not in your video. So my question is
a) is there any stetting in hardware bios?
b) Do I have to follow this guide pve.proxmox.com/wiki/Pci_passthrough and do something
c) Do I have to create linux bridges for additional NICs?
One person had the following method to solve similar issue but not sure as my one if not specific to NICs but for all PCI devices
create the directory /etc/default/grub.d
create a file ending with “.cfg” in that directory.
add the following to it:
Code:
GRUB_CMDLINE_LINUX_DEFAULT="${GRUB_CMDLINE_LINUX_DEFAULT} intel_iommu=on"
run update-grub.
Rebooted and everything works- see here
Kindly advise what I need to do. Thanks
I had the same issue with IOMMU, guessing because I am using a Core i7 and not a Xeon processor. I just followed instructions on pve.proxmox.com/wiki/Pci_passthrough. Solved that problem . Had the issue @vfxfan had as well, so just followed instructions as listed. Guessing when I do PCI passthrough it is passing through the entire card, therefor passing through all the NICs together :)
Thank you! Good call out. You do need to enable IOMMU. I skipped over that. Sorry!
Hi, I can start pfsense without adding the 2 pci card, when I add the pci card I get "TASK ERROR: start failed: QEMU exited with code 1"
Might be late to the party, followed your video and worked perfectly (thank you) only thing is if I reboot the vm (for pfsense) I don't get a WAN ip back, only way to get it is to reboot the Proxmox server, can't find anything to point me to the correct direction
is there a way to make segmentized network inside of virtualized firewall? i mean to deliver tagged vlans to pfsense or in my case Sophos XG Home firewall, through truenas (in my case Scale) thanks :)
How does this work when you want to use this as your main router from your ISP. Currently I have a consumer router that allows me to connect to proxmox. How do I swap over to pfsense by plugging my wan port into proxmox and having my lan still work. Do I need to configure proxmox in the same address space as my current router?
Hi i have try but pfsense give ip to my devices but they can not connect to the internet do you have any clue why? thanks
Hi Nice Vid!! At 3.48 you mention that you can pass-through of a 4 nic card only the 1/4 portion of it?? How is that possbile? I am used to Unraid on which you need to exclude the specific pci device you want to pass first and afterwards to give it to the VM.
Even more difficult if that device is a motherboard controller (usb, nic). Is it possible in Proxmox to pass-through motherboard controllers without braking things? Isn t in Proxmox mandatory the passed through device to be in its own iommu (so iommu capable motherboard needed?)
Last but not least did you have to put your isp's modem in bridged mode in order for this to work?
Thank you
ierosgr you can split up IOMMU groups in Proxmox with config! Not sure about splitting integrated motherboard items. Didn’t have to do anything different for my ISP’s modem. A NIC is a NIC to them.
@@TechnoTim At the start of the video you show up an intel nic which is an external pci device. Afterwards show you passing through a broadcom (so an intergraded one) Which of the two did you pass to Pfsense? Why a nic card differ from a gpu device and you dont have to enter conf files to exclude it from Hypervisor at boot like a gpu?
Hi Tim. You need to put a space before 'Techno' for the link to the HP Dual Gigabit NIC so the link works.
Tnks for the help, @Techni Tim!
If anyone get a error like this -> "TASK ERROR: KVM virtualisation configured, but not available. Either disable in VM configuration or enable in BIOS." - Please, follow this steps to solve!
Bye!
Hi Techno Tim. I've tried to setup pfSense as you di but everytime i start the pfSense VM i've Lost access to proxmox even if i have the server connect with a cable to LAN access with the ISP modem/router and a cable of my PCIE card connected with the WAN of the ISP. I've tried also to check the Netscape guide to virtualize pfSense on Proxmox but in that case pfSense can't take any WAN IP. Any suggestion on what to do?
If I do this and my pfsense VM doesn't boot - can I still access proxmox?
Just did this but having issues connecting to the internet. WAN dhcp is enabled and pulls an IP but nada. Any ideas? I did have IOMMU issues went through all that and got that resolved. I have a four port pci e nic that is being used for my wan and lan. I can access the gui but no internet.
Bro!, IPv6 Config please, is a head ache for me, I have only the lan IPV6 working GW is not working
Could I connect a switch from NIC to add more physical devices ?
I did not virtualize my router because the router has to be dedicated to it's own machine and I repurposed my Nuc because it has two Intel Nics and this machine is my slowest computer. Go Intel!
Hey question, what is the red ethernet cable in the back of your virtualization hosting server at 2:21 for?
Can I do it with Proxmox Virtual Network? I don't have a network card to add extra. Thanks for the video
Hello, nice video ! How do you connect other physical PCs to that virtualized router ?
How will all the virtual servers get ip? Also the proxmox will have a different network? Hiw will the whole thing work
When did Johnny Depp get into IT?
if i get this right, the one that goes out is goin into a switch, than its going back into my server (motherboard) lan, so my other vms have internet access as well.
Hello i got an issue, my dhcp not work on physical computer, only on virtual machine, can u help please
Great Video - first TechnoTim I have seen. Great job explaining and sharing. I have been using pfSense about 2 years now on an HP t620+ ThinClient with an added 2-port Intel i350-T2 card. Been working great, but I have this awesome Workstation class machine I want to use for ProxMox. I have 8.0.9 installed there, and I am just beginning. I purchased a 4-port i350-T4V2 for this box, and it is working fine. In the t620+ I had disabled the on-board NIC as was not using it.
I know that ProxMox requires a NIC for accessing the host/dashboard, but can it be one of the 2-ports I will use on the i350-T4? I have a cable from Cable modem to port 0 on the 4-port and cable from port 1 to the Netgear Orbi (wifi AP)...as it has a satellite in the other end of the house where the office is - so that I have Wired (per se) access back there and wifi is stronger. From the Orbi (at the ProxMox box & modem - there is a cable into the on-board NIC of the ProxMox host). If I unplug this, I lose access to the host dashboard.
Hi Tim, great tut. Had to do some IOMMU separation to get it to work but finally did it and working. Now, I have PFsense running inside vm giving its own network and dhcp to everything comming out through the lan port. So far so good. I want now to place the proxmox host behind pfsense as well and leave the primary modem only passing traffic to pfsense with DMZ. I just need to plug the nic (using proxmox) to the switch but before change de ip address? I'm not sure how to do this.
If Pfsense is running within Proxmox and connect to a modem, isn't promox exposed to the internet. Is it safe to just reroute all the traffic go through Pfsense including the other VM and CT?
7:01 how do you know what your LAN should be? I understand the WAN, but not how the LAN was wrong.
I have a error if I try to start pfsense in proxmox. Is there a way to solve this. :(
kvm: -device vfio-pci,host=0000:08:00.0,id=hostpci1.0,bus=pci.0,addr=0x11.0,multifunction=on: vfio 0000:08:00.0: device is already attached
TASK ERROR: start failed: QEMU exited with code 1
Hello, my networking setup at home are ONT and a openwrt router.
Can i set the pfsense on the midle of the ont and router
I'm sure it's been covered (in fact I know of 1 other creator that has) but running Unraid on Proxmox, I followed his skim-through and I can see it in the console but cant connect. Maybe in it elaborate on selecting network interfaces (cards) to split them among the chassis (Proxmox) and vms (PfSense, Unraid, and TrueNAS at least)
And longshot but if you have a multi-day chassis (like my sc846) how to specify specific bays to certain vms (not specific drives, that way any drive inserted into "bay 20" will be assigned to vm X.
would be helpful if you went through IOMMU and PCI passthrough for those NIC cards to be accessed by the VM
Check out my gpu passthrough video, same process!
@@TechnoTim if u blacklist your ethernet cards like with GPU does that mean other VMs don't get internet? this process doesnt seem straightforward and i cant find alot of resources online for NICs
My concern would be latency and in particular erratic latency. My router and gateway are dedicated purpose built hardware. Clearly I'm not undermining your video or intention, just a side note. I play twitch (FPS) games where a stable low latency is king. Great video 👌
Thanks for the comment! My pings are pretty low and consistently low (26 ms to Overwatch servers, 40 ms for Apex).All connections are physical since I am passing through the hardware to the VM and everything else is in memory.
@@TechnoTim .. It's all the background tasked to the CPU in a VM, I'm not even remotely suggesting it can't or shouldn't be done. Maybe a comparison video 😎👍
Thanks for the reply and good luck on your channel, interesting videos.
Thank you for the suggestion!
@@DJaquithFL I could answer some of this.
Most routers will use very lower power and small amounts of RAM, you would in most commercial devices have a 400Mhz CPU and around 128MB Ram. therefore the foot print of this is small on a "VM" providing that the node is not over loaded, modern and typically not slow or performance degraded via other software running on the same hardware.
@@guywhoknows .. Just my observations .. I have the best experienced when I'm at home alone not sharing our 500/500 Mbps connection even with a QoS.
FPS gaming is about milliseconds (ms), priority and not losing packets in UDP. Example of 2 people shoot simultaneously: 1. Lost Packets are the same as you not shooting; 2. Latency spikes up even a few milliseconds the server reads as your death, first shot wins.
Now if you're streaming e.g. Netflix via TCP .. latency spikes, lost packets, buffer bloat, etc are not noticeable and therefore irrelevant.
Hi TechnoTim, I hope you are able to answer one silly question about this setup: When experimenting with different virtualised router OSes I find the default LAN networks vary from product to product. And I like to just use the defaults most of the time in case changing them gives unexpected problems. This gives me a quandary about where to put my PVE management interface. I prefer to put it on the LAN, but that means it invariably ends up on a network number different from whatever I'm running for a router. So I have no access unless I mess with my network settings on my PC. Then I have to change them back to test out the router behaviour. I just wondered how you manage this problem in your setup, or do you just live with it?
could VMs connect to other physical servers in a lan network with pfsense ?!!
i got further with 8.0 then others version with this guide ty i have an older intel dual 100 nic that i may use as new is not in the cards yet lol.
I've tried this guide twice now, and it bricks my Proxmox file system every time forcing fresh install. I think it's either the VirtIO Block setting, or adding the PCIE card directly. Could you explain those more?
Any thoughts on installing with zfs? Seems to be the default these days
but how do the other vmsget internet access? do u use another port on ur pc and connect it to your switch/pfsense lan?
I was considering virtualising pfsense in proxmox but concerned about exposing my proxmox machine directly to the internet by placing it upstream of the firewall it is virtualising. Is this a concern? How did you mitigate this please. Thanks
Hi, Do you know how I could enable my computers that obtains network access from the virtual pfsense to be able to reach the proxmox servers webinterface? So for example if I have a virtual PC hosted on proxmox and its network is obtained from the virtual pfsense, how would I then allow for that virtual PC to be able to reach the proxmox interface?
@Techno Tim Thank you for the great video! I'm just scoping out the work I have a head of me, and want to know, can you access the proxmox UI via web from an IP dealt by the pfsense VM? Ideally i would like proxmox to be accessible from the virtual router, instead of the physically accessing the proxmox service with a keyboard and mouse. So my usecase is simple: access proxmox from my desktop that is connected to my virtual pfsense router.
Very helpful video, thanks! I have a question though if you don’t mind! Say i create a linux bridge to the passed-trough LAN port to allow connectivity between my other VMs and the physical switch managed by pfsense. Will the VMs bypass the pfsense firewall? Or will they be routed trough it? Thanks!
For starters this router Software going to cause some headache, especially for those must log into the isp via pppoe. In that case, if you have no ip on wan during first configuration of the interfaces, doesnt matter, you have to log into the web configurator and configure your wan interface, if still no change is detected, swap your cables and done, but with only two interfaces, if you cant get into the web ui, you have to swap cables and done, configure your wan. After all, i saw a mistake in the video, he accidentally or not, passed the same pci device to the vm twice, secondly you not really explained why you choosed the gateway to be on the ip 11. So this can be a little confusing.
what budget hardware do you recommend i buy so i can build a hardware pfsense to use as a firewall. i don't want to virtualize it. gonna use it at home non commercial use.
Check out all the gear I recommend here l.technotim.live/gear
I guess it's time to smash my buggy tplink router and say hello to virtual router. Cool tutorial as always. Keep it up man 👍
‘Atta boy!
how about showing us how to setup pfsense in proxmox and using that vm as the router for a cluster
Thanks for the video! If dont have a 2 port NIC can I add an additional 1 port NIC to go along with the built in one on my mobo?
Yes
Yes but then your Proxmox server loses it's connection as the VM will take both NICs as soon as it's started.
So if your server is virtualizing your router, how would you connect the server to the internet? Like, how would the other VMs on the server have access to the internet? Would you have to plug the ethernet port on your motherboard into the switch that your NIC is plugged into?
All machines on the same network would get an IP address from the virtualized pfSense from DHCP, which means it would also do NAT.
Just be sure that gateway it hands out is your pfSense server. It works like anything else!
@@TechnoTim Hi Tim, thanks for answering my question on such an old video! I'm in the middle of setting up my first virtualization server and your videos have been an invaluable resource. You've definitely earned a subscriber for life! 🙌🙌🙌
i now can run all my pfsnese on one server having over 10 ips and wanting firewall protection for all was a headache. now with all of them on one machine i can monitor them easier than before. Just a note if running nic that have 4 ports i didn't check the all function it would disable the 4 port to a 2 port for some weird reason.
Какое преимущество по сравнению с mikrotik ?
Hello Tim. I'm banging my head around a setup with a Chinese 6x ports firewall, proxmox and a openwrt lxc. I can't see LuCi, and the structure of CIDR and gateway is obscure to me. Is there a forum I can meet with you and have a chat? Are you on the proxmox forum?
Discord is the best place! Links in description and lots of community help there.
Sure would be nice to have pfSense as an LXC instead of a VM.