diff --git a/_posts/2024-04-17-docker-routing.md b/_posts/2024-04-17-docker-routing.md index d0c8f53..be34eac 100644 --- a/_posts/2024-04-17-docker-routing.md +++ b/_posts/2024-04-17-docker-routing.md @@ -1,7 +1,7 @@ --- title: Mosh + Tmux + Copy Paste layout: post -date: '2024-04-17 19:47:39 -0500' +date: "2024-04-17 19:47:39 -0500" description: Oracle Cloud Docker No Route To Host comments: true author: Andrew M. Zhang @@ -12,19 +12,20 @@ to behave in a typical fashion. # The Setup -We're using an Oracle Cloud Ubuntu VM as our host. Assume a machine with LAN address `10.0.0.100` on the `etho0` interface. -We're interested in running a Linuxserver.io Swag container as a reverse proxy on a user-defined bridge Docker network +We're using an Oracle Cloud Ubuntu VM as our host. Assume a machine with LAN address `10.0.0.100` on the `etho0` interface. +We're interested in running a Linuxserver.io Swag container as a reverse proxy on a user-defined bridge Docker network `lsio` that shows up in `ifconfig` as `br-xyz`. In the Swag container's compose file, we port forward `10.0.0.100:80:80` -and `10.0.0.100:443:443`. +and `10.0.0.100:443:443`. ```bash -# This is your typical linuxserver.io swag reverse proxy setup. +# This is your typical linuxserver.io swag reverse proxy setup. docker ps --format "table {{.ID}}\t{{.Names}}\t{{.Image}}\t{{.Ports}}\t{{.Networks}}" CONTAINER ID NAMES IMAGE PORTS NETWORKS 91b38a3164f2 swag lscr.io/linuxserver/swag 10.0.0.100:80->80/tcp, 10.0.0.100:443->443/tcp lsio ``` If we've set this up correctly, we expect to be able to reach a webpage at `http://10.0.0.100:80`. + ```bash ubuntu@myhost:~$ curl http://10.0.0.100:80 @@ -39,26 +40,28 @@ ubuntu@myhost:~$ curl http://10.0.0.100:80 # The Problem If we exec into the `swag` container, we expect the following ip and port to be reachable. Unfortunately, it is not. + ```bash ubuntu@myhost:~$ docker exec -it swag /bin/bash root@133d975caa2f:/# curl 10.0.0.100 curl: (7) Failed to connect to 10.0.0.100 port 80 after 2 ms: Couldn't connect to serve ``` -The reason for this is that Oracle VMs are pre-configured with a bunch of iptable rules and with UFW disabled. I believe -the reasoning for these rules is to ensure VMs are sufficiently hardened and also to manager their boot volume traffic, +The reason for this is that Oracle VMs are pre-configured with a bunch of iptable rules and with UFW disabled. I believe +the reasoning for these rules is to ensure VMs are sufficiently hardened and also to manager their boot volume traffic, which is mounted via iSCSI(?). They document their odd setup [here](https://blogs.oracle.com/developers/post/enabling-network-traffic-to-ubuntu-images-in-oracle-cloud-infrastructure). Note that Oracle states that you need to explicitly open port 80 for web traffic to flow. However, we did not do this with our Ubuntu VM, so why is traffic available at `10.0.0.100:80`? This behavior is driven by how Docker port forwards work. Documented [here](https://docs.docker.com/network/packet-filtering-firewalls/#docker-and-ufw): -> When you publish a container's ports using Docker, traffic to and from that container gets diverted before it goes -> through the ufw firewall settings. Docker routes container traffic in the nat table, which means that packets are -> diverted before it reaches the INPUT and OUTPUT chains that ufw uses. Packets are routed before the firewall rules can +> When you publish a container's ports using Docker, traffic to and from that container gets diverted before it goes +> through the ufw firewall settings. Docker routes container traffic in the nat table, which means that packets are +> diverted before it reaches the INPUT and OUTPUT chains that ufw uses. Packets are routed before the firewall rules can > be applied, effectively ignoring your firewall configuration. If we take a look at the NAT table, we see that traffic is DNAT'd through to our docker container. When the packet hits the DNAT rule, the next chain that will be executed is the FORWARD chain where it hits out `swag` reverse proxy. + ```bash ubuntu@myhost:~$ sudo iptables -L -v -t nat Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) @@ -84,7 +87,7 @@ Chain DOCKER (2 references) ``` This means you don't actually need to open up port 80 and 443 for the reverse proxy to work. Note however that traffic -originating from `br-xyz` network will not get DNAT-ed. Instead the `DOCKER` chain returns and packets end up in the +originating from `br-xyz` network will not get DNAT-ed. Instead the `DOCKER` chain returns and packets end up in the `INPUT` chain. Given Oracle's restrictive iptable settings, our packet gets rejected as it only matches the last rule. ```bash @@ -102,23 +105,24 @@ Chain INPUT (policy ACCEPT 0 packets, 0 bytes) # The Solution -For the typical reverse-proxy application, you probably won't need to access the `swag` reverse-proxy from inside the +For the typical reverse-proxy application, you probably won't need to access the `swag` reverse-proxy from inside the user-defined bridge network through ip port `10.0.0.100:80`. If the reverse-proxy redirects traffic from some public -ip or domain, you should be able to get the same effect by going through `http://mydomain.example.com:80`. However, +ip or domain, you should be able to get the same effect by going through `http://mydomain.example.com:80`. However, sometimes my reverse proxy is hosted on a Tailscale IP and it might need to "talk to itself" through its own Tailscale IP. An example would be if you ran an uptime-kuma service that needs to check if services are up behind your reverse proxy -hosted on your host's tailscale ip. +hosted on your host's tailscale ip. The solution is to allow traffic headed to port 80 and 443 in the INPUT chain. + ```bash # Add the following to your iptable, where br-xyz is the network your container is hosted -A INPUT -i br-xyz -d 10.0.0.100 -p tcp -m state --state NEW -m multiport --dports 80,443 -j ACCEPT ``` -I'm not aware of any way to automatically represent the IP addr of `eth0` instead of hardcoding it like I did above. You +I'm not aware of any way to automatically represent the IP addr of `eth0` instead of hardcoding it like I did above. You could make the ip subnet range by replacing it with something like `10.0.0.100/24`. ## IPv6 Oracle doesn't bork ip6tables, so you shouldn't need to do anything extra to get this to work. You will need to configure -docker to work with ipv6. \ No newline at end of file +docker to work with ipv6.