Going IPv6-Mostly with Tayga NAT64 on OPNsense
HTML-код
- Опубликовано: 12 июл 2024
- Today I'm going to expand on my previous IPv6-only experiments and try to move to an IPv6-mostly network, a few devices at a time.
But for this to work well, I need to give my IPv6-only clients access to the IPv4 internet, via NAT64. So join me as I setup Tayga to provide NAT64 functionality on OPNsense!
And once I have NAT64 setup, I can start to migrate clients entirely away from legacy networking. Some of them are relying on DNS64, and others on a 464xlat CLAT. Apple is the best at supporting this, but if you have a lot of mobile devices on your network, you can probably reduce the size of your DHCP pool significantly with very little effort.
Support me on Ko-Fi if you enjoy my content and find it useful:
ko-fi.com/apalrd
Feel free to chat about my upcoming projects on Discord!
/ discord
Timestamps:
00:00 - Introduction
01:21 - Transition Technology
07:31 - Tayga Setup
22:18 - Clients
28:18 - CLAT 
Наука
also on this episode, an impressive knowledge of a niche topic.
9:25 It's actually more complicated that that: The pf Firewall on FreeBSD (what OPNsense is based on) was originally developed and ported from OpenBSD. The modern OpenBSD version of the pf Firewall actually does support NAT64, but the differences between FreeBSD and OpenBSD pf are too karge because of things like multi processor support.
The FreeBSD ipfe Firewall, as you said, also supports NAT64, but it isn't recommended to use pf and ipfw on the same system at the same time, as afaik both run in the kernel.
It's a mess. I'm not sure how anyone thought that having 3 competing firewalls in the kernel in same project was a good idea.
@@apalrdsadventures II wouldn't call ipf a competitor to pf and ipfw because nobody really uses it. Most people use pf because of its simple and "human readable" syntax, while a smaller percentage of people use ipfw because it is a bit faster and more powerful at the cost of a harder syntax and lack of state synchronization with a second host (like pfsync).
I get a /48 IPv6 subnet from my ISP, so I am very happy. It makes IPv6 stuff lot easier on my home network. But it could be cool to try Tayga in my IPv4 only home lab to learn somethin new.
464xlat in android TV can be mixed bag. Had a smart TV that struggle handling ipv6 only network. I conclude that TV had slow cpu and just moved that device to ipv4 enabled subnet.
cool t-shirt :)
Great Video! Do you get a static ipv6 prefix from your provider? I would like to do my internal dns for my homeservices with the public ipv6 addresses but i only get a dynamic ipv6 prefix. So I guess I need to stick with ULAs for that usecase.
It's dynamic but never changes.
Sadly I found out yesterday my ISP does not provide ipv6 address in my area. I had assumed it was a configuration issue with my older UBNT router. I have fibre, but my ISP was a vary early adopter of it, and it was installed ten years ago, so the hardware may not support it. I'm in the slow process of upgrading all of my network gear. Including moving from that ER-6 router to OPNSense, and now would have bean a good time to learn ipv6.
Yeah, EdgeRouters can do dhcpv6-pd (IPv6 prefix delegation over DHCPv6) just fine, I use it on my ER-4. It is a bit annoying because the GUI doesn’t really support IPv6 config so you have to do it in the command line or config tree, and you have to set up separate firewall rules for IPv6, also on the command line.
If you want to run IPv6 you can use something like the free Hurricane Electric Tunnel Broker. I'm sure there are other options, too.
I really do think IPv6 Mostly is the future. Microsoft recently had a survey on IPv6 and seems to want to know what their customers want.
So are making progress.
Very cool you start work on an eBPF implementation !
25:18 yes, Facebook worked hard on that and I think Google did too, Amazon... not sure.
Meta is at the point where their express backbone (private fiber network) is strictly v6-only and they aren't giving v4 addresses on any links of their core network any more, only on loopback interfaces on routers and load balancers, using v6 next-hops.
Google isn't as public on what they are doing, as far as I've heard. Microsoft ran out of 10/8 internally and is trying to migrate to v6-only on their corporate network, but it sounds like it's taking some time for them.
@@apalrdsadventures also Github recently had an outage which on their outage description it said it was in part related to their work for IPv6.
yeah, someday Github will figure themselves out
@@apalrdsadventures I guess my comment is actually shadow banned, because I don't even see my comment anymore
I can still see it
Which devices don't support option 108? Are they IoT devices? Can you do this on pfsense? How do VLANs work with IPv6? What about ASICs and network switch chips, those route wirespeed for macs and VLANs, do any route between VLANs based on IP, and if so would any be ready for IPv6?
So:
- you don't need to be v6-only to support v6, they can coexist (as dual stack). Devices get both, and use both.
- In general option 108 is relatively new. On my network, only Apple devices actively request IPv6-only, but I don't own any Android devices, and they *should* support this too. That doesn't mean devices won't work IPv6-only, just they aren't actively requesting it.
- Most of my devices support at least v4+v6 correctly, even IoT ones, but not all. In particular, a few chinese IP cams I use for 3d printers and some smart plugs, at least on my network.
- New IoT gadgets are rapidly supporting v6 properly because it's required for the Matter / Thread standard in home automation. In fact, Matter hubs will create a v6 network on the local link if there isn't one, just for the automation devices to talk to each other.
- pfSense has the same ISC/Kea issue as OPNsense but you can add option 108 manually as well. They don't have a Tayga package though, so that's not nearly as easy to support.
- VLANs are a layer 2 thing, not layer 3, but a given vlan can have multiple subnets (including v4 and v6) running on it
- (L2) Switches don't care about v6 since that's layer 3. They are more likely to treat v6 multicast as broadcast.
- (L3) switches/routers can all route v6 in hardware at this point, if they can do v4 in hardware.
Now if only my isp (century link) would enable ipv6 support... Supposedly there have a 6rd setup, but it doesn't work with their hardware.
CenturyLink does and has supported IPv6 for many years now
@@bcm50 In Minnesota it's only via 6rd it seems, and it seems only with your own hardware. I'd love to be wrong and get it all working.
@@AndrewFrink Are you a residential customer?
@@bcm50 yes
Lumen (formerly CenturyLink) is the largest ISP in the world by ASN customer cone. They absolutely support v6 as a company, but being so large I'm sure they are extremely dysfunctional internally and can't figure out how to deploy v6 consistently across their many units.
Will this work with Plex Media Server? Can a remote ipv4 only user be able to see the PMS remotely?
Not directly, if Plex is v6-only. Tayga does support static mappings, but OPNsense doesn't have a way to configure that. The Tayga setup in OPNsense is only configured todeal with v6 -> v4 connections.
As there are so many comments about ISPs not supporting / enabling IPv6, I think it would be interesting to use something like a VPS server and route IPv6 over a VPN like wireguard. Does this make sense?
Yes. Only downside is Wireguard doesn't HW offload, so it can be slower than AES based alternatives if both sides support AES-NI.
@@apalrdsadventures wiregaurd is way faster on ARM devices
Depends on if your device is decent enough to have the arm v8 crypto extensions.
Some rough numbers on x86 are that the crypto alone is 10x slower doing ChaCha than aes-ni, but doing aes in software is another 10x slower than doing ChaCha in software, with vector extensions. Wireguard makes up some of this by not implementing any features and doing the bare minimum, and residing in the Linux kernel space, but IPsec is still faster on hw with aes accel (even though it kinda sucks, especially key exchange). On macOS and windows where it’s not in kernel the performance is way worse.
@@apalrdsadventures Well, IPsec wasn't built with performance in mind; it was built by complete paranoids. :D
IPsec really has two protocols though, the key exchange is done out of band in IKE and that’s where most of the complexity is.
Does the PBS machine bios include wake on alarm? You could use that instead of WOL.
no, the BIOS is pretty awful
So for this, you still need a static IPv6 prefix? I get a dynamic /56 from my ISP. Unless of course, you can somehow use a variable in the Tayga Prefix and IPv6 config sections?
If you use 64:ff9b as the translation prefix, and you can use 2001:db8 (the documentation prefix) or a ULA prefix for the Tayga ICMP address, the public prefix doesn't matter.
Please excuse me if you have already covered this. If I am dependent on my ISP giving me a subnet then if they change something on their network, don't I have to change all my internal addresses?
ISPs are not supposed to change prefixes, and decent ones will only change their prefix if they rearchitect their network (not just rebooting equipment at either end). But this is what Track Interface and SLAAC are for, to deal with prefix changes gracefully.
@@apalrdsadventures I know this has been mentioned by my employer as a reason we won't be switching to IPv6. We can't have any devices change addresses outside of our control.
Most ISPs will statically (on their end) allocate a /48 to business connections, just like they do with statically assigned IPv4s to business users. So only residential users even need to worry about it.
If it's a big deal to use an ISP's address range though, it's really cheap to buy your own IPv6 provider independent space. More hassle to do BGP on your own though, if you aren't already doing that for v4.
I actually wished ULA was an option for these things. It could have been a useful tool for these situations.
The trick is to move to zero trust networking and don't do firewalling in the world based on prefixes.
And an other advantage of not relying on the prefixed anymore, I haven't tried this with any every day network yet, but my guess is you can connect multiple ISPs and have failover in case of failures. I really need to test this some day, it seems like it should be possible.
@@apalrdsadventures"it's really cheap to buy your own IPv6 provider independent space", please do tell ???
What is this for a keyboard with display you are using?!?
It's the Kwumsy K3 - kwumsy.com/products/kwumsy-k3-touch-expanding-screen-keyboard?ref=EeoX5XElS_V68N
He has a video showcasing it.
The video EVERYONE should be doing but nobody else on this platform seems to care!
Now I can finally fix my new home network and bring it to its optimal state :)
Question: What about those smarty-pants ISPs that delivers current addressing but every time the sun shines, they change the prefix over and over?
Consider a non-hipothetical scenario where there aren’t other options available for a particular customer and he/she still wants to run IPv6-mostly.
So the 64:ff9b prefix isn't part of your delegation so it doesn't matter if the prefix changes. For Tayga, only the two IPs which are used to source ICMPs are in the prefix, and you can use 2001:db8 for those and it will function. ICMPs are only sent back upstream to NAT64 clients using this IP, so those two IPs won't make it onto the internet anyway.
@@apalrdsadventuresOK, good to know! Tks :)
@15:46 - what other address could you used for "IPv6 Prefix" instead of 64:ff9b::/96
64:ff9b::/32 is reserved for this purpose. Any /96 will work, even one from your ISP prefix. (OPNsense does not auto-fill the delegated prefix in this case, you have to type in something statically). Technically the prefix can be a few different lengths but practically most software only supports /96.
There are different opinions on using 64:ff9b vs a 2xxx prefix. Per the RFCs you cannot use 64:ff9b and also provide access to RFC1918 space, and Tayga will enforce that in code, but no other NAT64 solution is so picky as far as I know.
@@apalrdsadventures so, at this point in time, a fully IPv6 only network or domain is not possible?
1. I was hoping for a fully IPv6 only Windows domain/network, with the option for IPv6-to-IPv4 bet at the firewall only.
A v6-only Windows domain should be perfectly fine, it shoudn't need to deal with v4 for anything internal. You might have issues with non-Microsoft software not properly supporting v6 sockets with NAT64/DNS64 (Steam used to be the poster child for this problem, but they recently fixed this issue).
The only issue with using 64:ff9b::/96 with Tayga is that it won't translate private IPv4 addresses, but that doesn't matter if everything in your domain supports v6-only.
Would you make a video on using DNS with IPv6? I cannot for the life of me figure out how to set up DNS with IPv6 on my local network. If I use SLAAC I get random addresses afaik. Also my provider hands out dynamic prefixes on every router reboot, so I can't assign manual address with my prefix.
So DHCPv6 doesn't really solve the problem either, since most DHCPv6 hosts are privacy-aware enough to not give out their hostname as a unique identifier (usually the DUID is used instead), and the standards heavily discourage using hostnames in DHCPv6 due to issues with hostname-collision based attacks/bugs in DHCPv4.
Some other things that do solve the problem:
- Using EUI64 (MAC-based) instead of Stable Privacy addresses means the suffix will stay the same, and only the prefix needs to be added. Stable Privacy addresses are computed by taking the sha1 or sha256 of a number of factors including the network prefix, so the suffix will change on a new prefix. I use EUI64 on servers on my network usually.
- Use tokenized addresses (which are manually set with `ip token` in Linux). In Linux this basically replaces the eui64 address with your own suffix and otherwise uses the eui64 logic, so it fills in the prefix from RAs and you can rely on the suffix being constant.
In either case, the DNS server needs to replace the prefix when the WAN prefix changes. I do this manually (it's less often than yearly for me), but I could make a video on scripting it.
At least for me, domain joining is something I’d absolutely avoid
Mdns is a completely valid solution for small networks as well.
This is one of the many issues I've run into getting a sane ipv6 config set up for my network. Are there solutions for static addressing? Yeah but they require a lot of extra configuration and, depending on the choice, client side configuration. Whereas ipv4 just works. Ipv6 privacy features aren't relevant when you're not exposing everything to the internet, which also means these issues don't exist for local devices where you need it
The solution is to rely on DNS instead of caring what the individual addresses are.
I can't see any reason for corporate IT to go IPv6 internally, there doesn't appear to be any compelling reason. But on the Interest we are short, so the Internet should go IPv6 and be translated down at the router to the enterprise. Many existing devices don't support IPv6.
You can't go v4 clients -> v6 internet feasibly, so saying the internet should go v6 means everyone needs to have v6 to access the internet.
In a decently big business v6 solves a lot of network scaling problems like not having to 'right-size' subnets and routing between sites (which might have RFC1918 overlap in v4), but only if you don't also have to deploy v4 at the same time.
@apalrdsadventures parts of the Internet are using IPv6 now, and we can access them fine with our IPv4 homes, routers, and ISPs who haven't implemented IPv6. For home you use a tiny part of a single class C subnet, and it doesn't matter at all if we all use the same range, and we do, because of NAT, and that is how it's going to remain, at least for the next decade. Too much fuss, and no compelling reason, to change that. Big corporations have no interest in IPv6 internally either, I work in medium enterprise with just 1k servers, but have worked in places with many thousands, and tens of thousands of users. If there is an actual compelling argument to go IPv6 on every device on the planet I haven't heard it. A huge amount of kit doesn't work with IPv6, or doesn't implement it properly, yet. How will it make everybody's life better? Most people still don't know there are numbers behind the names. I'd love to see obvious benefits, tangible, at the moment I don't see it.
If you only have IPv4, then you can't access a server that doesn't provide a public v4 interface. Most hyperscalers are now running v6-only internally but still have v4 load balancers to deal with clients who haven't upgraded yet, so you can connect either way. We aren't yet at a point where anyone notable is dropping v4 support entirely, but some experimental sites are trying it. Running v6 doesn't have to mean getting rid of v4, but the biggest savings on operational efficiency come when you can do that also.
For home users, supporting v6 on your network results in lower latency (you're not going through NAT on your end and the v4->v6 gateway on their end), more so when dealing with CGNAT, and also no NAT traversal problems with any peer to peer apps or games since there is no NAT to traverse. Users are happy when these things work, and don't care which protocol they are using to make it happen.
For medium sized businesses, you probably are using the 10/8 space and probably splitting it up by location, and then more finely into individual subnets for different purposes. If you guess wrong on the subnet size, you waste more of 10/8, and even though it has 16M addresses, you burn a bunch of them with network + broadcasts if you subnet too finely and have a nightmare of a routing table on your IGP if you don't have a hierarchical nature and aggregate between sites. When you run out, you have to NAT between your own internal networks, and then you have to deal with the ambiguity of if an address in your logs came from one side of NAT or the other, and if you can even get to some resources through the NATs.
Then when you want to move from a 'traditional' 3-layer switched topology (using primarily vlans) to a routed topology (using primarily subnets), you have to split up all of those v4 subnets further, and burn a whole bunch of addresses on point to point links and networks / broadcasts of the new subnets and router loopbacks out of your 10/8 space. The network gets more efficient since you can deal with multipath better at L3, but now your v4 subnetting plan is not very happy with you.
Depending on how many users you have, you'll also likely need a decently large public v4 pool to deal with all of the inbound/outbound connections and every v4 has a real $ cost today. If you are relying on cloud services heavily, those v4 $ costs just went up a lot at AWS. If you do any sort of public SaaS stuff, you'll need a big v4 pool for your customer access as well, and you'll waste some of those expensive public addresses on your peering links and routers. Every customer that accesses your network over v6 means you can shrink the size of your v4 pools and v4 load balancers and sell off more addresses. Every external site your internal users access over v6 means your NAT appliances get smaller and become less of a network sizing bottleneck.
When you get to the ISP or large business scale, you desperately want everyone to be using v6 so you can avoid going too far down the CGNAT hole, which results in angry customers who can't get their xbox to work. There are no v4 addresses left, so you can't get more when your customer base expands, and have to continually squeeze the ones you have harder over time. As customers adopt v6, this pushes load off the CGNAT gateways, which unlike v6 routers need to maintain a lot of state and logging for legal reasons.
Of course there are legacy devices in any network, but every subnet that has v6 enabled means a decently large portion of the traffic going to the internet will use it (even if the internal resources don't), reducing demand for NAT and public v4s. Every subnet that can transition to v6-only means more 10/8 space is freed up in the internal address plan, so you can continue to grow without being limited by NAT capacity.
@@apalrdsadventures dude had no retort to this lol
1 hour = 3600 seconds
ur right, I was thinking of 86400 (24 hours)
you should try to run your own open source v of discord - my thoughts, also a request to do a special on updated hardware - ws/nas etc - as always thanks for the good content - i think ipv6 is basically a bit too complex for the vast majority but tools to make things easier to convert should beep popping up - i mean it took you a year to really make the jump and you are having a slow migration - this shows how it is tough even for admins and tech savvy - it just isn't really ready for prime time yet despite the obvious need to convert due to shrinking availability of ipv4 #dual stack #split horizon plaese try to set up a dedicated ids/ips to do 24/7 pkt cap and maybe even try to incorp polar proxy or sslstrip proxy to examine all ssl/tls pkts both optimized nas and also ids have big demand for smb sector - maybe even do some HA - opnsense ha would be a fun one for you to delve into and it isn't super complicated once you get into it - people need protection from downtime, maybe do HA plus dual wan for agg plus failover? #triwan #bump list #round robin #load balancing
What do you run OPNsense on?
Currently have a Protectli FW4B as 'primary' and a FW4C as my test system, but I eventually plan on swapping the two.
@@apalrdsadventures Those look really nice. Thanks for the info!
Ipv6 is simple. More simple than V4. Making it work with v4 is not.
the correct pronunciation would be taigA
Is useless, is amazing that most ISPs don't provide FIXED IPV6 addresses, whats the point? And on top of that most only provide /64 IPV6 IP, so no subnet is allowed, for god sake I just give up. At least in IPV4 I can have as much subnets as I want. Tunneling was not an option either, as hurricane dosen't has servers in my country putting my latency over the roof, so yeah, IPV6 is not ready yet, at least here.
Guidance to ISPs from RIPE and ARIN is a /48 for business and /56 for fixed residential customers. I believe the 'scarcity' issue is partially APNIC's fault, if you're in the Asia-Pacific region.
@@apalrdsadventures No, Im in Europe, my ISP dosen't even understand what I m talking about, these guys are completely culess in IPV6, I had to warn them that they had all ports 443 and 80 open from the WAN side in all clients Routers on IPV6, basically they are not ready, try cal 2 different ones, same story, they only have dynamic IPV6 addresses to offer, whats the point, gazillions of IPV6 and only have dynamic? WTF? I just give up because I don't have time to waste on a configuration that should take 20 minutes to make, they are simple not ready.
An interesting map - stats.labs.apnic.net/ipv6/XE - huge difference between western/northern and eastern/southern Europe.
@@apalrdsadventures ok with this is explained, my country is only 8% IPV6, yeah is going to take time.
Most* smaller ISPs tend to roll out v6 when they are otherwise forced into CGNAT on v4. CGNAT gateways are not a small expense for an ISP (if they are doing it properly). I know a lot of ISPs do this poorly, but a proper CGNAT gateway should be either allocating individual customers to static public IP:port range blocks (i.e. you get 6.6.6.6 and ports 2000 - 3999) or logging every IP:port pair used for legal reasons - ISPs need to be able to forward legal requests back to the subscriber. Usually CGNAT gateways also sync session tables across multiple units for redundancy, although I know a lot of smaller ISPs skip out on this. It's also a traffic funnel in a network, since all user v4 traffic has to go through the gateway.
Going v6 for an ISP means their CGNAT gateways see less traffic (= smaller CGNAT hardware) and they can allocate less ports to each subscriber (= less $ for IPv4 space), since the big CDNs all support v6 and a lot of user bandwidth will be to those CDNs over v6.