Kong proxy Container pod restarts with PostgreSQL timeout error when retrieving server_version_num

[mayankmurari]

We have deployed Kong as Active-Active in two Data-Centers where we are running Kong with 5 pods in each DC. From both instances of Kong we are connected to same Postgres through HAProxy where postgres is also running with leader node and multiple replicas for high availability.

During a recent issue we observed all Kong proxy container restarted in both the DCs due to a timeout when attempting to retrieve the PostgreSQL server version number. From Postgres side we see it was up. The following error is logged in Kong:

ingress-controller container logs -

time="2025-02-25T19:28:16Z" level=error msg="checking config status failed" error="making HTTP request: Get \"[https://localhost:8444/status\":](https://localhost:8444/status/%22:) dial tcp 127.0.0.1:8444: connect: connection refused"

proxy container logs -

nginx: [warn] the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /kong_prefix/nginx.conf:8 2025/02/25 19:28:16 [warn] 1#0: the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /kong_prefix/nginx.conf:8 nginx: [error] init_by_lua error: /usr/local/share/lua/5.1/kong/init.lua:660: [PostgreSQL error] failed to retrieve PostgreSQL server_version_num: timeout stack traceback: [C]: in function 'assert' /usr/local/share/lua/5.1/kong/init.lua:660: in function 'init' init_by_lua(nginx-kong.conf:42):3: in main chunk 2025/02/25 19:28:19 [error] 1#0: init_by_lua error: /usr/local/share/lua/5.1/kong/init.lua:660: [PostgreSQL error] failed to retrieve PostgreSQL server_version_num: timeout

Our understanding is while Kong pods are up and running and in case of any connectivity issues with Postgres, Kong should remain up and there should not be any impact to live traffic.

So need help in understanding why Kong proxy containers would have restarted?

Kong version: 3.7.1
Deployment: Kubernetes