Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--http-bind-port=80 does not work. binds on 8080 #589

Closed
aep opened this issue Nov 14, 2023 · 12 comments
Closed

--http-bind-port=80 does not work. binds on 8080 #589

aep opened this issue Nov 14, 2023 · 12 comments
Assignees

Comments

@aep
Copy link

aep commented Nov 14, 2023

hi,
i'm using the helm chart with useHostNetwork on bare metal to expose haproxy directly on the internet

helm install haproxy-controller --namespace haproxy-controller haproxytech/kubernetes-ingress --set controller.service.type=LoadBalancer --set controller.kind=DaemonSet --set controller.daemonset.useHostPort=true --set controller.daemonset.useHostNetwork=true --set controller.containerPort.http=80  --set controller.containerPort.https=44

which starts kubernetes-ingress with

    Image:         haproxytech/kubernetes-ingress:1.10.9
    Image ID:      docker.io/haproxytech/kubernetes-ingress@sha256:412f54ca550d17c990d7a6afadd6c7fdb2008057a95577b12b1a9c6906878102
    Ports:         80/TCP, 443/TCP, 1024/TCP
    Host Ports:    80/TCP, 443/TCP, 1024/TCP
    Args:
      --default-ssl-certificate=haproxy-controller/haproxy-controller-kubernetes-ingress-default-cert
      --configmap=haproxy-controller/haproxy-controller-kubernetes-ingress
      --http-bind-port=80
      --https-bind-port=443
      --ingress.class=haproxy
      --publish-service=haproxy-controller/haproxy-controller-kubernetes-ingress
      --log=info
      --prometheus

however, this doesnt actually make the ports work. it listens on 8080 anyway.

crictl exec -ti c3a27c6e50a47    cat /etc/haproxy/haproxy.cfg 

frontend https
  mode http
  bind 127.0.0.1:8080
@fabianonunes
Copy link
Contributor

This bug was addressed in #568, but the it hasn't been released yet. You can give the nightly tag a try (--set controller.image.tag=nightly).

@aep
Copy link
Author

aep commented Nov 14, 2023

thanks. nightly doesnt appear to listen to anything

# _version=1
# HAProxy Technologies
# https://www.haproxy.com/
#
# this file is not meant to be changed directly
# it is under haproxy ingress controller management
#

global
  daemon
  master-worker
  pidfile /var/run/haproxy.pid
  stats socket /var/run/haproxy-runtime-api.sock level admin expose-fd listeners
  default-path config

defaults
  timeout http-request 5000
  timeout connect 5000
  timeout client 50000
  timeout queue 5000
  timeout server 50000
  timeout tunnel 3600000
  timeout http-keep-alive 60000

peers localinstance
   peer local 127.0.0.1:10000

frontend https
  mode http
  http-request set-var(txn.base) base
  use_backend %[var(txn.path_match),field(1,.)]

frontend http
  mode http
  http-request set-var(txn.base) base
  use_backend %[var(txn.path_match),field(1,.)]

frontend healthz
  mode http
  monitor-uri /healthz
  option dontlog-normal

frontend stats
   mode http
   http-request set-var(txn.base) base
   http-request use-service prometheus-exporter if { path /metrics }
   stats enable
   stats uri /
   stats refresh 10s


[WARNING]  (213) : config : frontend 'https' has no 'bind' directive. Please declare it as a backend if this was intended.
[WARNING]  (213) : config : frontend 'http' has no 'bind' directive. Please declare it as a backend if this was intended.
[WARNING]  (213) : config : frontend 'healthz' has no 'bind' directive. Please declare it as a backend if this was intended.
[WARNING]  (213) : config : frontend 'stats' has no 'bind' directive. Please declare it as a backend if this was intended.
[WARNING]  (213) : config : Removing incomplete section 'peers localinstance' (no peer named 'uca2k').
[NOTICE]   (213) : New worker (253) forked
[NOTICE]   (213) : Loading success.

@fabianonunes
Copy link
Contributor

During the initial initialization, it won't listen on any port. However, after the first fork, it will then select the ports you have chosen.

@aep
Copy link
Author

aep commented Nov 14, 2023

i'm not sure what that means but the helm chart install a health check, which is failing because the stats endpoint isnt up, so the container is being deleted

@aep
Copy link
Author

aep commented Nov 14, 2023

  Warning  Unhealthy  7m6s (x20 over 7m25s)   kubelet            Startup probe failed: Get "http://10.182.192.1:1042/healthz": dial tcp 10.182.192.1:1042: connect: connection refused
  Normal   Killing    7m6s                    kubelet            Container kubernetes-ingress-control
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 01-aux-cfg: executing... 
[cont-init.d] 01-aux-cfg: exited 0.
[cont-init.d] done.
[services.d] starting services
Memory limit for HAProxy: 6618MiB
[services.d] done.
Memory limit for Ingress Controller: 3309MiB
2023/11/14 12:29:20 maxprocs: Leaving GOMAXPROCS=64: CPU quota undefined
2023/11/14 12:29:20 
 _   _    _    ____
| | | |  / \  |  _ \ _ __ _____  ___   _
| |_| | / _ \ | |_) | '__/ _ \ \/ / | | |
|  _  |/ ___ \|  __/| | | (_) >  <| |_| |
|_| |_/_/   \_\_|   |_|  \___/_/\_\\__, |
 _  __     _                       |___/             ___ ____
| |/ /   _| |__   ___ _ __ _ __   ___| |_ ___  ___  |_ _/ ___|
| ' / | | | '_ \ / _ \ '__| '_ \ / _ \ __/ _ \/ __|  | | |
| . \ |_| | |_) |  __/ |  | | | |  __/ ||  __/\__ \  | | |___
|_|\_\__,_|_.__/ \___|_|  |_| |_|\___|\__\___||___/ |___\____|


2023/11/14 12:29:20 HAProxy Ingress Controller dev dirty
2023/11/14 12:29:20 Build from: 
2023/11/14 12:29:20 Git commit date: 
2023/11/14 12:29:20 ConfigMap: haproxy-controller/haproxy-controller-kubernetes-ingress
2023/11/14 12:29:20 Ingress class: haproxy
2023/11/14 12:29:20 Empty Ingress class: false
2023/11/14 12:29:20 Publish service: haproxy-controller/haproxy-controller-kubernetes-ingress
2023/11/14 12:29:20 Using local backend service on port: 6061
2023/11/14 12:29:20 Default ssl certificate: haproxy-controller/haproxy-controller-kubernetes-ingress-default-cert
2023/11/14 12:29:20 Frontend HTTP listening on: 0.0.0.0:80
2023/11/14 12:29:20 Frontend HTTPS listening on: 0.0.0.0:443
2023/11/14 12:29:20 Controller sync period: 5s

2023/11/14 12:29:20 Running on uca1k
[NOTICE]   (212) : haproxy version is 2.8.3-86e043a
[WARNING]  (212) : config : frontend 'https' has no 'bind' directive. Please declare it as a backend if this was intended.
[WARNING]  (212) : config : frontend 'http' has no 'bind' directive. Please declare it as a backend if this was intended.
[WARNING]  (212) : config : frontend 'healthz' has no 'bind' directive. Please declare it as a backend if this was intended.
[WARNING]  (212) : config : frontend 'stats' has no 'bind' directive. Please declare it as a backend if this was intended.
2023/11/14 12:29:20 k8s/main.go:95 [] Running on Kubernetes version: v1.28.3 linux/amd64
[WARNING]  (212) : config : Removing incomplete section 'peers localinstance' (no peer named 'uca1k').
2023/11/14 12:29:20 INFO    k8s/main.go:182 [] Global CR defined in API core.haproxy.org/v1alpha2
2023/11/14 12:29:20 INFO    k8s/main.go:182 [] Defaults CR defined in API core.haproxy.org/v1alpha2
2023/11/14 12:29:20 INFO    k8s/main.go:182 [] Backend CR defined in API core.haproxy.org/v1alpha2
2023/11/14 12:29:20 INFO    builder.go:220 [] running controller data server on :6060, running, prometheus
2023/11/14 12:29:20 INFO    builder.go:244 [] running default backend server on :6061
Error :  failed to initialize haproxy API client: dial unix /var/run/haproxy-runtime-api.sock: connect: no such file or directory
2023/11/14 12:29:20 PANIC   src/main.go:124 [] failed to initialize haproxy API client: dial unix /var/run/haproxy-runtime-api.sock: connect: no such file or directory
[NOTICE]   (212) : New worker (252) forked
[NOTICE]   (212) : Loading success.
Memory limit for Ingress Controller: 3309MiB
2023/11/14 12:29:21 maxprocs: Leaving GOMAXPROCS=64: CPU quota undefined
2023/11/14 12:29:21 
 _   _    _    ____
| | | |  / \  |  _ \ _ __ _____  ___   _
| |_| | / _ \ | |_) | '__/ _ \ \/ / | | |
|  _  |/ ___ \|  __/| | | (_) >  <| |_| |
|_| |_/_/   \_\_|   |_|  \___/_/\_\\__, |
 _  __     _                       |___/             ___ ____
| |/ /   _| |__   ___ _ __ _ __   ___| |_ ___  ___  |_ _/ ___|
| ' / | | | '_ \ / _ \ '__| '_ \ / _ \ __/ _ \/ __|  | | |
| . \ |_| | |_) |  __/ |  | | | |  __/ ||  __/\__ \  | | |___
|_|\_\__,_|_.__/ \___|_|  |_| |_|\___|\__\___||___/ |___\____|


2023/11/14 12:29:21 HAProxy Ingress Controller dev dirty
2023/11/14 12:29:21 Build from: 
2023/11/14 12:29:21 Git commit date: 
2023/11/14 12:29:21 ConfigMap: haproxy-controller/haproxy-controller-kubernetes-ingress
2023/11/14 12:29:21 Ingress class: haproxy
2023/11/14 12:29:21 Empty Ingress class: false
2023/11/14 12:29:21 Publish service: haproxy-controller/haproxy-controller-kubernetes-ingress
2023/11/14 12:29:21 Using local backend service on port: 6061
2023/11/14 12:29:21 Default ssl certificate: haproxy-controller/haproxy-controller-kubernetes-ingress-default-cert
2023/11/14 12:29:21 Frontend HTTP listening on: 0.0.0.0:80
2023/11/14 12:29:21 Frontend HTTPS listening on: 0.0.0.0:443
2023/11/14 12:29:21 Controller sync period: 5s

2023/11/14 12:29:21 Running on uca1k
[NOTICE]   (212) : haproxy version is 2.8.3-86e043a
[WARNING]  (212) : Exiting Master process...
[ALERT]    (212) : Current worker (252) exited with code 143 (Terminated)
[WARNING]  (212) : All workers exited. Exiting... (0)
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.

@fabianonunes
Copy link
Contributor

fabianonunes commented Nov 14, 2023

Increase the startup failure threshold (--set controller.startupProbe.failureThreshold=300). It's possible that the startup is too slow.

@aep
Copy link
Author

aep commented Nov 14, 2023

that didnt help. it just never starts listening on port 1042

@ivanmatmati
Copy link
Collaborator

Hi @aep , We'll have a look at your issue. Thanks for reporting.

@ivanmatmati ivanmatmati self-assigned this Nov 15, 2023
@ivanmatmati
Copy link
Collaborator

Hi @aep , can you specify the version of Helm Charts you're using ?

@ivanmatmati
Copy link
Collaborator

I've run some test with version 1.34.1 from Helm Charts, which should be correct according the controller version in your logs, and nothing wrong. I got:

frontend http
  mode http
  bind 0.0.0.0:80 name v4
  bind :::80 name v6
  http-request set-var(txn.base) base
  http-request set-var(txn.path) path
  http-request set-var(txn.host) req.hdr(Host),field(1,:),lower
  http-request set-var(txn.host_match) var(txn.host),map(/etc/haproxy/maps/host.map)
  http-request set-var(txn.host_match) var(txn.host),regsub(^[^.]*,,),map(/etc/haproxy/maps/host.map,'') if !{ var(txn.host_match) -m found }
  http-request set-var(txn.path_match) var(txn.host_match),concat(,txn.path,),map(/etc/haproxy/maps/path-exact.map)
  http-request set-var(txn.path_match) var(txn.host_match),concat(,txn.path,),map_beg(/etc/haproxy/maps/path-prefix.map) if !{ var(txn.path_match) -m found }
  use_backend %[var(txn.path_match),field(1,.)]
  default_backend haproxy-controller_default-local-service_http

frontend https
  mode http
  bind 0.0.0.0:443 name v4 crt /etc/haproxy/certs/frontend ssl alpn h2,http/1.1
  bind :::443 name v6 crt /etc/haproxy/certs/frontend ssl alpn h2,http/1.1
  http-request set-var(txn.base) base
  http-request set-var(txn.path) path
  http-request set-var(txn.host) req.hdr(Host),field(1,:),lower
...

BTW, I've noticed a typo in your command line, you missed the last number in port number. It should be 443 not 44.

@ivanmatmati
Copy link
Collaborator

Can you check the logs of your attempt with the official release ? Because it seems that you've got something wrong preventing the computation of the very first transaction and you're stuck with the default haproxy configuration file. If necessary increase the log level by adding the extra args --log=debug. You add it in your values.yaml at this place:

controller:
  extraArgs:
  - --log=debug

@ivanmatmati
Copy link
Collaborator

Closed because no activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants