In the case of learning, VLAN1 is fine. When your lab starts mimicking production, then it should match your production environment. When you get to that point, this video is beyond that scope.
I'd advise home folks and particularly businesses never assign CGNAT addresses to anything... It's not extra private IP space for your private network to use. It's specifically made for carriers to utilize. (I suppose you might argue for it in lab settings like Tom is showing but it's not needed)
@scbtripwire 100.64 /10 space... He mentions it on his VLAN spreadsheet briefly. If a carrier is using them for WAN transport or devices which provide internet but have routing protocols active... You can run into big issues
How/ why would it be any better in any way than just subnets? Asking seriously as a respectful networking idiot. Edit: doesnt it 'solve' the same problem as ipv6? I think I get why it would be silly to do anything but emulate an ISP. If you were to assign an ip and that comes up in some DNS resolution on any machine in your network, seems like it could resolve some site or cdn or something to something on your Lan
@@MaxUglyI’m nitpicking here - IPv6 doesn’t solve as much as it removes the need to solve for the fact that we ran out of addresses. NAT was a band-aid, the internet was always designed for host-to-host connection. But I’m being pedantic and your point is well understood. I agree with you on the second part, using a range that nobody’s ever supposed to use internally seems like a safe way to avoid issues in this case.
Since you say you should not route storage, how do you handle it when a VM needs access to e.g. an NFS share for docker? Do I also create that VLAN then on the virtualization host and give the VM an extra interface with a static IP just for accessing that share? Would be interesting to know since I plan on redoing my setup soon and the issue has come up
A storage network is usually not routed for two main reasons, the first is security the second because they have a different MTU size usually 9600, this will decrease the TCP overhead and improve performance because a packet is able to to handle a bigger payload. This technique is called “jumbo frames” since the packets are a lot bigger than normal packets, which have an MTU size of 1500 the router needs to partition the packets into several different packets before it can send them to the internet, since this is a performance wise very expensive operation this is usually disabled on the router. This phenomenon is called packet fragmentation, you can also see this on ppp and pppoe links when the MTU size is set too big.
@ I’m shocked you would cite jumbo frames as being a primary driver of keeping storage on the same subnet. The advantages of non-standard MTU are limited (in the case of 9000 versus 1500 MTU, less than 5% efficiency gain), the downsides are numerous, and in short I would say mostly it’s just not worth it. The original intent had to do with having to process overhead which can be done now with hardware offloading to the NIC. Anyway, Tom’s point had to do with unnecessarily burdening routers with traffic. He used the term “routing” specifically and accurately, as in, _on a different segment; needing to be routed_. Clearly it’s better to e.g. have multiple interfaces or IPs and connect your storage up to multiple network segments on local switches if need be rather than isolate them on their own subnet and require routers to route all traffic there constantly. I just didn’t understand his point because I assumed less accuracy in his wording than he, correctly, used.
@@LAWRENCESYSTEMS Interesting, thank you. Would using VLANs for that over the same interface work as well then when you don't have enough physical ports? I would setup a LACP bond on my TrueNAS box with the default VLAN for management and define a tagged VLAN for storage. They are 10G ports FYI.
This is a really great video and while watching I think I understood most of what was going on but feel like if I went to implement a similar setup I might get a bit lost on some of the smaller details. I would really like to see some reference doc or something similar linked to be able to refer back to with more details on the config and setup. If there is one and I missed where to find it my apologies in advance. I only recall mention of the diagrams on github. Keep up the great work as always! will continue to watch the channel love the videos.
Thanks! Another great video. Question: do you use VLANs at customer sites? I have walked into SMB customers that don't have any documentation on VLANs and I usually remove them. Thanks again
Bit of along shot here , but asking, can I use the unifi flex mini 2.5gb network switch, like a normal switch, or do I have to have other ubiquiti hardware for it to work ? Been trying to find answers on tinternet without any luck Cheers
I guess we could do the same thing but with a virtualized unifi controller instead of pfsense. I was thinking about doing this for using terraform to provision my network.
Do you have any videos on the "manamgment LAN"?...... What goes on it? How do you access device on there from your daily workstation/laptop? Which VLAN should you use for it?... if not VLAN 1, what do you do with VLAN 1?
As I said in the video, that is where I put XCP-ng hosts, the management for the TrueNAS Storage servers (VLAN 20 is where the storage lives). For access I have a Wireguard tunnel but OpenVPN or Tailscale could work just as well.
@@LAWRENCESYSTEMS I think he was asking for what you use to actually handle credentials, authentication and SSO. Not what software do you use for end users and password management.
I wish I could wrap my head around why VLAN tags are needed on top of using different IP ranges/subnets for things. I know I'm missing something, I just can't figure out what. 😆
Remember vlan segregate the network and they are function in layer 2 while subnets function in layer 3. If a network is separated by vlans it will need a different ip scheme which is why there are vlan tags that will move the traffic into the correct subnets
As stated vlans are layer 2. By default everything in the same subnet is switched. Your end device (pc etc) has arp, IP and MAC address, for all those systems in that subnet. The default gateway is used when you try to reach an ip not in that subnet. PFsense is the default gateway and router and firewall in this example. For security and performance you want to segregate traffic with PFsense. To separate that traffic on a switch you use virtual lans or vlans. PFsense understands those frames containing vlan information called dot1q. Without using vlans and dot1q tags all local ip traffic would be switched. And you couldn’t separate and control your traffic between storage and management, for example.
@@samsampier7147 Yeah, I know how VLANs work, I was pointing out that what he said previously just didn't add any clarity. Also I would argue that with subnetting you can just add authentication protocols on top of it to secure it as well as any VLAN setup, but probably VLANs are easier and require less infrastructure.
More top quality content from Tom!
Glad you like it!
In the case of learning, VLAN1 is fine. When your lab starts mimicking production, then it should match your production environment.
When you get to that point, this video is beyond that scope.
Thanks for this great video Tom. Will implement something similar when I upgrade my lab in the coming year.
I'd advise home folks and particularly businesses never assign CGNAT addresses to anything... It's not extra private IP space for your private network to use. It's specifically made for carriers to utilize. (I suppose you might argue for it in lab settings like Tom is showing but it's not needed)
CGNAT addresses? Which ones are those? I haven't reached the end of the video yet. Edit: Oh by CGNAT you meant non-private? Gotcha.
@scbtripwire 100.64 /10 space... He mentions it on his VLAN spreadsheet briefly. If a carrier is using them for WAN transport or devices which provide internet but have routing protocols active... You can run into big issues
@scbtripwire the 100 addresses he mentioned on his spreadsheet
How/ why would it be any better in any way than just subnets? Asking seriously as a respectful networking idiot.
Edit: doesnt it 'solve' the same problem as ipv6? I think I get why it would be silly to do anything but emulate an ISP. If you were to assign an ip and that comes up in some DNS resolution on any machine in your network, seems like it could resolve some site or cdn or something to something on your Lan
@@MaxUglyI’m nitpicking here - IPv6 doesn’t solve as much as it removes the need to solve for the fact that we ran out of addresses. NAT was a band-aid, the internet was always designed for host-to-host connection. But I’m being pedantic and your point is well understood. I agree with you on the second part, using a range that nobody’s ever supposed to use internally seems like a safe way to avoid issues in this case.
great content again Tom!
Glad you liked it!
@@LAWRENCESYSTEMS always do enough to keep TrueNAS in house! :)
That is the best title, My eyes went wide. Finally the secrets will be unlocked!
Had to rewatch a few times but I get it now. Great video.
I think I'd enjoy working for you, Tom.
Since you say you should not route storage, how do you handle it when a VM needs access to e.g. an NFS share for docker? Do I also create that VLAN then on the virtualization host and give the VM an extra interface with a static IP just for accessing that share? Would be interesting to know since I plan on redoing my setup soon and the issue has come up
@@colinschaffer320 yeah, I’m curious about the “don’t route storage” comment too.
Yes lawrence.video/storagedesign
A storage network is usually not routed for two main reasons, the first is security the second because they have a different MTU size usually 9600, this will decrease the TCP overhead and improve performance because a packet is able to to handle a bigger payload. This technique is called “jumbo frames” since the packets are a lot bigger than normal packets, which have an MTU size of 1500 the router needs to partition the packets into several different packets before it can send them to the internet, since this is a performance wise very expensive operation this is usually disabled on the router. This phenomenon is called packet fragmentation, you can also see this on ppp and pppoe links when the MTU size is set too big.
@ I’m shocked you would cite jumbo frames as being a primary driver of keeping storage on the same subnet. The advantages of non-standard MTU are limited (in the case of 9000 versus 1500 MTU, less than 5% efficiency gain), the downsides are numerous, and in short I would say mostly it’s just not worth it. The original intent had to do with having to process overhead which can be done now with hardware offloading to the NIC.
Anyway, Tom’s point had to do with unnecessarily burdening routers with traffic. He used the term “routing” specifically and accurately, as in, _on a different segment; needing to be routed_. Clearly it’s better to e.g. have multiple interfaces or IPs and connect your storage up to multiple network segments on local switches if need be rather than isolate them on their own subnet and require routers to route all traffic there constantly. I just didn’t understand his point because I assumed less accuracy in his wording than he, correctly, used.
@@LAWRENCESYSTEMS Interesting, thank you. Would using VLANs for that over the same interface work as well then when you don't have enough physical ports?
I would setup a LACP bond on my TrueNAS box with the default VLAN for management and define a tagged VLAN for storage. They are 10G ports FYI.
This is a really great video and while watching I think I understood most of what was going on but feel like if I went to implement a similar setup I might get a bit lost on some of the smaller details. I would really like to see some reference doc or something similar linked to be able to refer back to with more details on the config and setup. If there is one and I missed where to find it my apologies in advance. I only recall mention of the diagrams on github. Keep up the great work as always! will continue to watch the channel love the videos.
The diagram is on GtiHub but I am not sure what else you are looking for.
Good video, off topic question, are you going to review the new PFSENSE 24.11 update?
Thanks! Another great video. Question: do you use VLANs at customer sites? I have walked into SMB customers that don't have any documentation on VLANs and I usually remove them. Thanks again
Yes, we segment and isolate customer networks as needed.
Hey Tom, great video! In this video is the head end pfsense connected to your home pfsense? I didn’t see any home type vlans. Thanks!
Not for this lab server.
Bit of along shot here , but asking, can I use the unifi flex mini 2.5gb network switch, like a normal switch, or do I have to have other ubiquiti hardware for it to work ?
Been trying to find answers on tinternet without any luck
Cheers
All the UniFi access points and switches need their software controller ruclips.net/video/TmxFL02Gpl0/видео.html
@LAWRENCESYSTEMS thank you 👍
I guess we could do the same thing but with a virtualized unifi controller instead of pfsense. I was thinking about doing this for using terraform to provision my network.
Do you have any videos on the "manamgment LAN"?......
What goes on it?
How do you access device on there from your daily workstation/laptop?
Which VLAN should you use for it?... if not VLAN 1, what do you do with VLAN 1?
As I said in the video, that is where I put XCP-ng hosts, the management for the TrueNAS Storage servers (VLAN 20 is where the storage lives). For access I have a Wireguard tunnel but OpenVPN or Tailscale could work just as well.
Kinda off topic but how do you manage user credentials for pc login and network shares? Is that case by case basis or do you always run AD?
I use Bitwarden
@@LAWRENCESYSTEMS I think he was asking for what you use to actually handle credentials, authentication and SSO. Not what software do you use for end users and password management.
@@BreetaiZentradi We don't use SSO in the lab, but we do use Microsoft Entra ID for the business for SSO.
@@LAWRENCESYSTEMS I tend to lump SSO in with local auth.
I always use VLAN 69 for management 😎
I wish we could use vlan 80085
@Tom - Did you get permission from the Elders of The Internet to use a picture of The Internet in your diagram??? 😅
He copyrighted ©️ it 😂
Of course!
Is Tom employee of the month? He obviously got permission from "the hawk"
I wish I could wrap my head around why VLAN tags are needed on top of using different IP ranges/subnets for things. I know I'm missing something, I just can't figure out what. 😆
Remember vlan segregate the network and they are function in layer 2 while subnets function in layer 3. If a network is separated by vlans it will need a different ip scheme which is why there are vlan tags that will move the traffic into the correct subnets
@sherwinstone4876 Do you have any links to diagrams? I'm a visual person.
@@sherwinstone4876 coming from someone who understands VLANs - this doesn’t clear it up at all.
As stated vlans are layer 2. By default everything in the same subnet is switched. Your end device (pc etc) has arp, IP and MAC address, for all those systems in that subnet.
The default gateway is used when you try to reach an ip not in that subnet. PFsense is the default gateway and router and firewall in this example.
For security and performance you want to segregate traffic with PFsense. To separate that traffic on a switch you use virtual lans or vlans. PFsense understands those frames containing vlan information called dot1q.
Without using vlans and dot1q tags all local ip traffic would be switched. And you couldn’t separate and control your traffic between storage and management, for example.
@@samsampier7147 Yeah, I know how VLANs work, I was pointing out that what he said previously just didn't add any clarity. Also I would argue that with subnetting you can just add authentication protocols on top of it to secure it as well as any VLAN setup, but probably VLANs are easier and require less infrastructure.
If you could do something similar with 100% Unifi that would be aweeesome
You can't have a virtual UniFi firewall so that would be ruled out, I think the rest can be done.
#25! :)
Explicitly, not implicitly. 😉
words are hard
first!
Second 🥈