Load nf_tables in preparation of Alpine 3.19

[AaronDewes]

https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/47102 changes the default iptables backend to nf_tables. To prepare for the change (and to make sure it's not forgotten once DinD is updated), load nf_tables into the kernel.

[tianon]

This is unfortunately a bit more complicated because it's the host's opinion about iptables vs nftables that we need to respect here, and I think doing that correctly is going to require more coordination inside dockerd itself -- @thaJeztah, do you know if there's already a good issue/PR about this upstream? 😬

[tianon]

See canonical/docker-snap#68 for somewhere else this same problem has been discussed previously (but not officially with the upstream project AFAIK) 👀

[thaJeztah]

thanks for the ping; let me add @akerouanton and @robmry here

[tianon]

Ok, discussed with some other Moby maintainers inside Docker, and given that most users will be running this in a network namespace (and that's definitely recommended), the mismatch should be fine. That being said, I do think we should only load one or the other based on what our in-container userspace is going to actually use (which is knowable for us here), so I think we should both roll this into #461 and update it to swap ip_tables for nf_tables in this script (not just adding nf_tables). 🙇 ❤️

cc @jnoordsij 👀 ❤️ (happy to take over from here and push this all the way through if you'd prefer 👍)

I also wonder if we should, after loading the module, run something like the iptables command we gated this line on to begin with, but perhaps that's better left to dockerd failing, since it does fail appropriately and early. (Just thinking out loud and writing it down somewhere relevant. 😄)

[jnoordsij]

cc @jnoordsij 👀 ❤️ (happy to take over from here and push this all the way through if you'd prefer 👍)

@tianon I've rebased #461 to include this change and updated it in they way I think you intended to. Please check though, I'm still very much not fully aware of all the complications involved; also feel free to alter it further and finish things up!

[tianon]

Big thanks to all who've contributed to getting this through ❤️

[paolomainardi]

Today, we faced an issue on GKE (not autopilot) from a Gitlab CI runner using DIND on Kubernetes.

This was the issue:

runner-ykqmlyn3-project-832-concurrent-0-h3hlk7ma svc-0 failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to create NAT chain DOCKER: iptables failed: iptables -t nat -N DOCKER: iptables v1.8.10 (nf_tables): Could not fetch rule set generation id: Invalid argument

And this was the configuration:

services:
  - name: docker:24.0.7-dind
    command:
      [
        "--registry-mirror",
        "https://mirror.gcr.io",
        "--mtu=1300",
        "--network-control-plane-mtu=1300",
      ]

For now we have fixed reverting back to alpine 3.18.

[piyush-bhuwalka]

@tianon When would this change be available in docker:dind image ?

On using the latest one we are getting this issue

ip: can't find device 'nf_tables'

[manneorama]

This hit us this morning (GKE using action-runner-controller) using docker:dind as a sidecar. We worked around it by pinning the dind image to the previous version for now.

[pdbrito]

Ran into this today also, fixed it by replacing docker:24.0-dind with docker:24.0.7-dind-alpine3.18

[tianon]

Fixed by #465 which will be updated by docker-library/official-images#15921 👍