Interaction between Jool and XFRM #427
Comments
Solid starting point. The interface fields in the kernel packet structure have always seemed awkward to me, because whether they refer to the inbound or the outbound interface is context-dependent. But assuming Do you see a rule in
IPv6 finished successfully. Layer 2 likely failed to find a neighbor, and tried to respond an ICMPv6 error by way of a Layer 3 callback. The ICMPv6 error could not be sent either, seemingly because XFRM could not map one of the packets into a session. So the core issue does seem to be that it fails to find a neighbor after the packet has been dumped into the wrong interface. |
Initially I suspected the fact clients were given addresses from the LAN prefix could confuse Jool, so I set up a different prefix (not linked to any physical interface) and used that, but the packets still go to the LAN. Maybe the problem is that there is no static route? I think the packet are routed only based on this policy: |
|
Please bear with me, because I don't really have any experience configuring VPNs, and in its fervor to sell them to me in the name of buzzwords, the Internet is remarkably atrocious at explaining them. How come everyone involved seems to be in the same IPv4 network (128.66.0)? Is "IPv4 host" also supposed to be a VPN client? Is 128.66.0 supposed to be the VPN? Is What is the point of the translator in this environment? Which is What's the expected packet flow? It seems to be
I don't know the internals of XFRM, but if the packet is supposed to be "routed based on its policies," I'd expect the policies to result in routes in the routing table. It shouldn't matter if the route is static or dynamic. Are you getting an empty table when you execute How do you read this? I'm guessing this means "if you get packet |
Sorry, I just made the addresses up and it must have become confusing.
No, the VPN is strictly IPv6-only.
It performs NAT64 for the IPv6-only LAN, and ideally also the VPN clients. The gist is, the VPN client should be able to also reach IPv4 hosts through
I'll draw you a diagram: The expected flow for: is the following:
Essentially yes: there is only the /64 LAN and the default route via the ISP.
Correct. |
|
Ok, this makes much more sense now. Well, looking at the diagram here, it seems XFRM happens after Prerouting. If you compare this to Jool's version, it doesn't really fit. Jool picks up your packet at Prerouting, then dumps it in Postrouting. At that point, the packet might U-turn into the "Xfrm encode" box, but by then it skipped all the XFRM lookup analysis. This matches your
Jool left the packet in Your non-translated packet, by contrast, traverses "Forward" as normal:
I think your best bet is to separate Jool and XFRM into separate network namespaces. That way, you'll have complete control over what happens when. Would this be viable? |
Oh ok, so it means Jool and XFRM are essentially incompatible, correct?
I'll have to look into this as I've never used network namespaces. From what I understand I need to put Jool into an ad-hoc namespace, create a veth device that links it to the real namespace and route 64:ff9b::/96 to it. Anyway, thank you for your time and for Jool! |
Assuming the "Xfrm lookup fwd policy" and "Xfrm lookup out policy" boxes (from the thermalcircle.de diagram) are mandatory: If they're in the same namespace, yes. Unforturately.
Yes. You'll also need to route the pool4 address to it. Alternatively, if you do IPv4 in one interface and IPv6 in a separate interface, you can move one of those interfaces to the virtual namespace. This might simplify the routing, as it'll mean packets won't have to do U-turns in and out of the Jool namespace: I don't know if NixOS has a special way of doing it, but here's how I setup temporary (won't survive reboots) virtual namespaces in one of my test suites:
To move interface |
|
Alternatively alternatively, place Jool in one machine, and XFRM in another. At the end of the day, that's what the namespaces would be simulating. |
|
Unfortunately I can't move the IPv4 WAN interface into the jool namespace, because it's used by the VPN and other services too. I've implemented the setup from this script I found and it seems to work, but it's quite ugly (the IPv4 NAT in particular). Do you happen to have a cleaner setup? Thank you again. |
|
Well, I suppose I would need to know why you feel it's ugly. That NAT most likely exists to hide the "private" IPv4 mini-network formed between the Jool and the global namespaces. (192.0.2 in the diagram below.) If you want to ditch the NAT, one way is to grant public IPv4 interfaces to (By the way: You should probably replace 192.0.2 with 192.168.0, because the former is supposed to be a documentation network.) |
Well, it's the bad old IPv4-IPv4 translation that breaks connectivity: |
I'm running Jool in NAT64 mode on a router that is also the endpoint of an IPsec VPN in tunnel mode.
The VPN clients get IPv6 addresses from a local subnet and are NDP-proxied, so behave like they were on the LAN (encrypted traffic comes in from the IPv4 WAN and comes out either to the IPv6 WAN or LAN decrypted). Everything works as expected, except NAT64.
To see what goes wrong I traced a single ping from a VPN client to an IPv4 host on the internet:
So, the echo request is correctly translated, the echo reply is received but apparently is not translated back to IPv6.
However, using this eBPF tool I can see the translated IPv6 reply has in fact been crafted but is being dropped for some reason:
For comparison, this is what the trace of a ping to a host that doesn't need translation looks like:
I don't know how to interpret these traces because I have no knowledge of the netfilter internals, but clearly the packet written by Jool is being sent to a different (wrong?) interface (LAN instead of WAN).
I'm not sure whether this issue is a limitation of Jool, Jool and XFRM clashing or the VPN policies not being configured correctly. Do you have any insight?
The Jool configuration is simply:
{ "framework": "netfilter", "global": { "pool6": "64:ff9b::/96" }, "instance": "default" }If you're interested, the VPN setup is fully defined here
The text was updated successfully, but these errors were encountered: