Skip to content

Commit

Permalink
Reformats code with prettier
Browse files Browse the repository at this point in the history
  • Loading branch information
@andrewmzhang
andrewmzhang committed Apr 28, 2024
1 parent 36c80b5 commit 3868d2d
Showing 1 changed file with 20 additions and 16 deletions.
36 changes: 20 additions & 16 deletions _posts/2024-04-17-docker-routing.md
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Mosh + Tmux + Copy Paste
layout: post
date: '2024-04-17 19:47:39 -0500'
date: "2024-04-17 19:47:39 -0500"
description: Oracle Cloud Docker No Route To Host
comments: true
author: Andrew M. Zhang
Expand All @@ -12,19 +12,20 @@ to behave in a typical fashion.

# The Setup

We're using an Oracle Cloud Ubuntu VM as our host. Assume a machine with LAN address `10.0.0.100` on the `etho0` interface.
We're interested in running a Linuxserver.io Swag container as a reverse proxy on a user-defined bridge Docker network
We're using an Oracle Cloud Ubuntu VM as our host. Assume a machine with LAN address `10.0.0.100` on the `etho0` interface.
We're interested in running a Linuxserver.io Swag container as a reverse proxy on a user-defined bridge Docker network
`lsio` that shows up in `ifconfig` as `br-xyz`. In the Swag container's compose file, we port forward `10.0.0.100:80:80`
and `10.0.0.100:443:443`.
and `10.0.0.100:443:443`.

```bash
# This is your typical linuxserver.io swag reverse proxy setup.
# This is your typical linuxserver.io swag reverse proxy setup.
docker ps --format "table {{.ID}}\t{{.Names}}\t{{.Image}}\t{{.Ports}}\t{{.Networks}}"
CONTAINER ID NAMES IMAGE PORTS NETWORKS
91b38a3164f2 swag lscr.io/linuxserver/swag 10.0.0.100:80->80/tcp, 10.0.0.100:443->443/tcp lsio
```

If we've set this up correctly, we expect to be able to reach a webpage at `http://10.0.0.100:80`.

```bash
ubuntu@myhost:~$ curl http://10.0.0.100:80
<html>
Expand All @@ -39,26 +40,28 @@ ubuntu@myhost:~$ curl http://10.0.0.100:80
# The Problem

If we exec into the `swag` container, we expect the following ip and port to be reachable. Unfortunately, it is not.

```bash
ubuntu@myhost:~$ docker exec -it swag /bin/bash
root@133d975caa2f:/# curl 10.0.0.100
curl: (7) Failed to connect to 10.0.0.100 port 80 after 2 ms: Couldn't connect to serve
```
The reason for this is that Oracle VMs are pre-configured with a bunch of iptable rules and with UFW disabled. I believe
the reasoning for these rules is to ensure VMs are sufficiently hardened and also to manager their boot volume traffic,
The reason for this is that Oracle VMs are pre-configured with a bunch of iptable rules and with UFW disabled. I believe
the reasoning for these rules is to ensure VMs are sufficiently hardened and also to manager their boot volume traffic,
which is mounted via iSCSI(?). They document their odd setup [here](https://blogs.oracle.com/developers/post/enabling-network-traffic-to-ubuntu-images-in-oracle-cloud-infrastructure).
Note that Oracle states that you need to explicitly open port 80 for web traffic to flow. However, we did not do this
with our Ubuntu VM, so why is traffic available at `10.0.0.100:80`? This behavior is driven by how Docker port forwards
work. Documented [here](https://docs.docker.com/network/packet-filtering-firewalls/#docker-and-ufw):
> When you publish a container's ports using Docker, traffic to and from that container gets diverted before it goes
> through the ufw firewall settings. Docker routes container traffic in the nat table, which means that packets are
> diverted before it reaches the INPUT and OUTPUT chains that ufw uses. Packets are routed before the firewall rules can
> When you publish a container's ports using Docker, traffic to and from that container gets diverted before it goes
> through the ufw firewall settings. Docker routes container traffic in the nat table, which means that packets are
> diverted before it reaches the INPUT and OUTPUT chains that ufw uses. Packets are routed before the firewall rules can
> be applied, effectively ignoring your firewall configuration.

If we take a look at the NAT table, we see that traffic is DNAT'd through to our docker container. When the packet hits
the DNAT rule, the next chain that will be executed is the FORWARD chain where it hits out `swag` reverse proxy.
```bash
ubuntu@myhost:~$ sudo iptables -L -v -t nat
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
Expand All @@ -84,7 +87,7 @@ Chain DOCKER (2 references)
```
This means you don't actually need to open up port 80 and 443 for the reverse proxy to work. Note however that traffic
originating from `br-xyz` network will not get DNAT-ed. Instead the `DOCKER` chain returns and packets end up in the
originating from `br-xyz` network will not get DNAT-ed. Instead the `DOCKER` chain returns and packets end up in the
`INPUT` chain. Given Oracle's restrictive iptable settings, our packet gets rejected as it only matches the last rule.
```bash
Expand All @@ -102,23 +105,24 @@ Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
# The Solution
For the typical reverse-proxy application, you probably won't need to access the `swag` reverse-proxy from inside the
For the typical reverse-proxy application, you probably won't need to access the `swag` reverse-proxy from inside the
user-defined bridge network through ip port `10.0.0.100:80`. If the reverse-proxy redirects traffic from some public
ip or domain, you should be able to get the same effect by going through `http://mydomain.example.com:80`. However,
ip or domain, you should be able to get the same effect by going through `http://mydomain.example.com:80`. However,
sometimes my reverse proxy is hosted on a Tailscale IP and it might need to "talk to itself" through its own Tailscale
IP. An example would be if you ran an uptime-kuma service that needs to check if services are up behind your reverse proxy
hosted on your host's tailscale ip.
hosted on your host's tailscale ip.
The solution is to allow traffic headed to port 80 and 443 in the INPUT chain.
```bash
# Add the following to your iptable, where br-xyz is the network your container is hosted
-A INPUT -i br-xyz -d 10.0.0.100 -p tcp -m state --state NEW -m multiport --dports 80,443 -j ACCEPT
```
I'm not aware of any way to automatically represent the IP addr of `eth0` instead of hardcoding it like I did above. You
I'm not aware of any way to automatically represent the IP addr of `eth0` instead of hardcoding it like I did above. You
could make the ip subnet range by replacing it with something like `10.0.0.100/24`.

## IPv6

Oracle doesn't bork ip6tables, so you shouldn't need to do anything extra to get this to work. You will need to configure
docker to work with ipv6.
docker to work with ipv6.

0 comments on commit 3868d2d

Please sign in to comment.