How do we expose additional TCP ports from a container in Kamal 2 using kamal-proxy?

[navinpeiris]

Hi all!

I've got a case where I need to expose some additional TCP ports from the container. It's an elixir app that needs to expose two ports for EPMD so that we can connect all the nodes in the cluster.

Currently, using Kamal 1.x, I'm able to do this using traefik labels, but going through Kamal 2 and kamal-proxy code, it wasn't clear if it was possible.

Cheers!

[geek411]

To expose additional TCP ports for EPMD in Kamal 2, you can use the accessories feature along with Traefik as a TCP proxy. Here's how to modify your config/deploy.yml:

• Add a Traefik container as an accessory to handle TCP routing.
• Configure Traefik to expose the necessary ports (4369 for EPMD and a range like 9000-9100 for node distribution).
• Add Traefik labels to your main service to enable TCP routing.

For example:

service: your_app_name
image: your_docker_image:tag

servers:
  web:
    hosts:
      - your_server_ip_or_hostname

health_check:
  path: /healthz
  port: 4000

env:
  clear:
    PORT: 4000

accessories:
  tcp_proxy:
    image: traefik:v2.5
    roles: [web]
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    command:
      - --providers.docker=true
      - --providers.docker.exposedbydefault=false
      - --entrypoints.web.address=:80
      - --entrypoints.epmd.address=:4369
      - --entrypoints.distribution.address=:9000-9100
    ports:
      - 80:80
      - 4369:4369
      - 9000-9100:9000-9100

traefik_labels: &traefik_labels
  - traefik.enable=true
  - traefik.http.routers.${SERVICE_NAME}.rule=Host(`${DOMAINS}`)
  - traefik.http.routers.${SERVICE_NAME}.entrypoints=web
  - traefik.tcp.routers.epmd.rule=HostSNI(`*`)
  - traefik.tcp.routers.epmd.entrypoints=epmd
  - traefik.tcp.routers.epmd.service=epmd
  - traefik.tcp.services.epmd.loadbalancer.server.port=4369
  - traefik.tcp.routers.distribution.rule=HostSNI(`*`)
  - traefik.tcp.routers.distribution.entrypoints=distribution
  - traefik.tcp.routers.distribution.service=distribution
  - traefik.tcp.services.distribution.loadbalancer.server.port=9000-9100

labels:
  <<: *traefik_labels

I hope that helps.

[navinpeiris]

Thanks for that @geek411!

In case it helps anyone else, this is the config that ended up working for me:

accessories:
  traefik:
    service: traefik
    image: traefik:v3.1
    roles:
      - web
    options:
      publish:
        - 4369:4369
        - 9100:9100
    cmd: "--providers.docker --providers.docker.exposedByDefault=false --entryPoints.epmd.address=:4369 --entryPoints.beam.address=:9100 --log.level=INFO"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"

labels:
  traefik.enable: true
  traefik.tcp.routers.epmd.rule: "ClientIP(`0.0.0.0/0`)"
  traefik.tcp.routers.epmd.priority: 5
  traefik.tcp.routers.epmd.entryPoints: epmd
  traefik.tcp.routers.epmd.service: epmd
  traefik.tcp.services.epmd.loadBalancer.server.port: 4369
  traefik.tcp.routers.beam.rule: "ClientIP(`0.0.0.0/0`)"
  traefik.tcp.routers.beam.priority: 5
  traefik.tcp.routers.beam.entryPoints: beam
  traefik.tcp.routers.beam.service: beam
  traefik.tcp.services.beam.loadBalancer.server.port: 9100

[sebastian]

This worked well for me too. Thank you for the setup!

It does however seem a bit unfortunate to have to run a separate service just to expose a TCP port that docker could easily have exposed straight out of the box... I wonder if there shoudn't be a way to instruct Kamal to pass custom settings straight to docker instead?

[sebastian]

Actually, on looking at the source code, it turns out you can pass custom config in Kamal 2.0.

servers:
  web:
    hosts:
      - 1.2.3.4
    options:
      publish: "8001:8001"

8001 being the port I have configured my nodes to listen to for incoming connections (and I am running without epmd). However this prevents the system from doing zero-downtime deployes as it cannot start the next version of the app while a previous container binds to the port... so I guess all this is to say: never mind... The traefik option still seems the way to go.

[konung]

Our app gets 99.99% for the traffic during “business hours”. We just need to expose ports, and do deployment overnight or on the weekends anyway. This will work well enough for us.

Thank you

[schrockwell]

Please note that you may also have to run kamal accessory start traefik to get it started.