Can't connect to same host agent with Docker Compose

[mauvehed]

Description

First off, I've read through #359 and tried many of the suggestions there.

I just installed bezel and have no issues with the docker compose deployment on other hosts. However, nothing I do will enable the Hub to see my local agent. I've deployed via the example compose file:

services:
  beszel:
    image: henrygd/beszel:latest
    container_name: beszel
    restart: unless-stopped
    extra_hosts:
      - host.docker.internal:host-gateway
    ports:
      - 8090:8090
    volumes:
      - /data/docker/beszel/data:/beszel_data

  beszel-agent:
    image: henrygd/beszel-agent:latest
    container_name: beszel-agent
    restart: unless-stopped
    network_mode: host
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      PORT: 45876
      # Do not remove quotes around the key
      KEY: "my key"

With this config I've tried host.docker.internal (as the docs suggest) and no go. The error in the logs is:

ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

However, checking my /var/log files, I see no errors nor attempted SSH connections. I also know the key works and have tested it from multiple hosts.

I tried configuring a docker network and using the container name, that also didn't work.

Last thing, I have no firewall configured. ufw is disabled, but I still tried adding 45876 to iptables allow for both 127.0.0.1 and 0.0.0.0.

No go on anything.

OS / Architecture

linux/amd64 (docker compose for hub & agent)

Beszel version

0.9.1

Installation method

Docker

Configuration

No response

Hub Logs

No response

Agent Logs

No response

[mauvehed]

Oh, I've tried connecting to the actual container to do investigation, but it always errors with sh or bash not existing.

OCI runtime exec failed: exec failed: unable to start container process: exec: "bash": executable file not found in $PATH: unknown

[henrygd]

I just added instructions to the docs on how to get a shell in the container. Let me know if it doesn't work for you.

This is a strange one because the error you provided in the first post suggests the hub resolved agent's SSH server but the key was denied. Please double check that the key has been updated in the compose file and the container has been restarted (maybe with --force-recreate).

I doubt the issue is firewall related, but please check #359, especially Fumler's comment at the bottom.

If it's still not working, try taking the agent container out of host network mode, so both containers are in the same docker network, and connect to it using the hostname beszel-agent. We should have an alternative to running the agent in host mode in the next release.

[paulverket]

Another data point. I tried running the hub and agent as specified in the quick start but the agent would never connect. ufw was disabled and telnet worked?? The logs kept showing Error(8) dial tcp [::1]:45876: connect: connection refused. Commenting out host mode fixed it.
Pi4 running Ubuntu 24.04
Docker version 26.1.3, build 26.1.3-0ubuntu1~24.04.1

[henrygd]

dial tcp [::1]:45876: connect: connection refused

Was this error showing in the hub? I'm not sure why it would be trying to use loopback. Are you sure you used host.docker.internal as the Host / IP when adding the system in the hub?

[paulverket]

My mistake. That error was showing up in the pocketbase logs. I used loopback when adding the system as opposed to host.docker.internal from the common issues page. Now I have network stats - yay.

[henrygd]

No worries, that part is confusing and I wish there was an easier way to handle it that didn't require running an additional container.