Unable to use Kamal2 with CF and Hertzner

[zaddy6]

# Name of your application. Used to uniquely configure containers.
service: cowriter-api

# Name of the container image.
image: xxx-service-staging

# Deploy to these servers.
servers:
  web:
    - xxx
    - xxx
  # job:
  #   hosts:
  #     - 192.168.0.1
  #   cmd: bin/jobs

# Enable SSL auto certification via Let's Encrypt (and allow for multiple apps on one server).
# If using something like Cloudflare, it is recommended to set encryption mode 
# in Cloudflare's SSL/TLS setting to "Full" to enable end-to-end encryption. 
proxy:
  ssl: false
  forward_headers: true
  # kamal-proxy connects to your container over port 80, use `app_port` to specify a different port.


# # Credentials for your image host.
registry:
  server: xxxx.dkr.ecr.us-east-1.amazonaws.com
  username: AWS
  password: 
    - KAMAL_REGISTRY_PASSWORD

# Configure builder setup.
builder:
  arch: amd64
  dockerfile: "./Dockerfile"
  context: "./"

env:
  secret:
    - MONGO_USERNAME

I have created a LB on Hertzner and added both servers, I have also added a proxied A record pointing to the LB server on cloudflare however I get a website down error

Although visiting the server IP for both the LB and Hosts works without issues

[tuladhar]

In which port is Hetzner LB listening on?
Are targets behind Hetzner LB healthy?
What's mode is SSL/TLS setting on Cloudflare?

[zaddy6]

1. LB is listening on port 80
2. Yes
3. Mode is set to Full (Not strict) on CF

[navidemad]

deploy.yml:

# Note: If using Cloudflare, set encryption mode in SSL/TLS setting to "Full" to enable CF-to-app encryption. 
proxy:
  ssl: true
  host: api.foobar.com
  # Proxy connects to your container on port 80 by default.
  app_port: 3000 # if you use thruster remove this line, if you don't use thruster set this line

You have then to setup DNS records with proxy toggle checked (A with ipv4 and AAAA with the ipv6 without the /64)
Then on cloudfare dns, you will have to create two records:
A with @ if it's the full domain, or just "api" if it's a subdomain, and as value you set your IPv4
AAAA with @ if it's the full domain, or just "api" if it's a subdomain, and as value you set your IPv6 (remove the /64

Then commit your changes, and kamal deploy.
Additionnal i add the gem "cloudflare-rails" to my production environement group in my Gemfile.

[zaddy6]

This config wouldnt work as SSL: True only works if you are deploying to a single server, in my case I am deploying to 3 servers, and then all 3 are under a single hertzner LB

[miharekar]

I'd try removing the proxy section as instructed:
https://github.com/basecamp/kamal/blob/main/lib/kamal/cli/templates/deploy.yml#L17

[zaddy6]

still doesnt fix the issue

[zaddy6]

only thing that seems to work is to set SSL as flexible on CF which isnt very secure

[sebastian]

I had the same issue.

I solved it by:

• disabling the proxy in the deploy config (proxy: false)
• letting the load balancer terminate SSL using a certificate and private key provided by Cloudflare

You can get the certificate and private key from Cloudflare by going to “SSL/TLS > Origin server”. The certificate and private key can then be used by the load balancer.

[bradpurchase]

This fixed it for me. Of course, I had to create a new certificate in Cloudflare to get a private key, but then I was able to input this in Security > Certificates within Hetzner.

[fabriciocarboni]

Here's where DHH explains.
https://youtu.be/QC4b2teG_hc?si=ioNQ45ogiWwSb0C5&t=1396

[sebastian]

... which clearly did not work for us, hence this issue.

In his video, DHH doesn't do any SSL termination in the load balancer (see this part of the video you linked to: https://youtu.be/QC4b2teG_hc?t=1332) hence it requires that you also have your Cloudflare account set to allow non-secure connections between it and your servers. If you, however, want SSL both between your users and Cloudflare as well as between Cloudflare and your servers (to avoid their credentials flying around in the open), then the you need to take other steps, namely having the Hetzner loadbalancer doing SSL termination too (see this: #1119 (comment)).