Issue with deployed rke2 second Ingress-Nginx Controller inside a Kubernetes cluster running on bare-metal. #7315
Replies: 1 comment
-
|
Hi @brandond We had helm chart yaml file of 1st ingress controller in below directory Initially we had started without changing ports but we were getting below error ["nodes don't have free ports for the requested pod ports"] so we changed port to use different ports then pods started showing in running state. Below is the content of yaml we used to deploy second nginx ingress controller apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
annotations:
helm.cattle.io/chart-url: https://rke2-charts.rancher.io/assets/rke2-ingress-nginx/rke2-ingress-nginx-4.10.101.tgz
rke2.cattle.io/inject-cluster-config: "true"
name: rke2-ingress-nginx-abcx3
namespace: kube-system
spec:
bootstrap: false
targetNamespace: abcx3apps
chartContent: <Copied from default RKE2 Ingress Controller/>
set:
global.clusterCIDR: 10.42.0.0/XX
global.clusterCIDRv4: 10.42.0.0/XX
global.clusterDNS: 10.XX.0.10
global.clusterDomain: cluster.local
global.rke2DataDir: /var/lib/rancher/rke2
global.serviceCIDR: 10.43.0.0/16
valuesContent: |-
controller:
allowSnippetAnnotations: true
ingressClass: nginx-abcx3
ingressClassResource:
name: nginx-abcx3
enabled: true
controllerValue: "k8s.io/ingress-nginx-abcx3"
ingressClassByName: true
watchIngressWithoutClass: false
config:
use-forwarded-headers: true
enable-real-ip: true
publishService:
enabled: true
containerPort:
http: 8082
https: 8083
hostPort:
enabled: true
http: 8082
https: 8083
admissionWebhooks:
enabled: true
port: 8084
service:
port: 8084
service:
enabled: true
type: LoadBalancer
external:
enabled: true
externalTrafficPolicy: Local
annotations:
metallb.universe.tf/loadBalancerIPs: 10.11.XXX.74Please let us know if anything we are missing. |
Beta Was this translation helpful? Give feedback.
-
What happened:
We have deployed two Nginx ingress controller in our cluster. The first ingress controller works fine but we are facing issue with second one. When accessing the service through second ingress controller (http://x3.abc.com/sample), we get This site can’t be reached x3.abc.com took too long to respond. error. We followed the same step as the first ingress controller including assigning different IP address using metallb and different ingress class in different namespace.
What you expected to happen:
For testing purpose we deployed a simple nginx image using standard yaml files. Everything is up and running, No error logs in pods, service mapped correct endpoints and ingress resource got external IP assigned by Second nginx ingress controller But when we try to access http://x3.abc.com/sample gives Site can't be reached error.
NGINX Ingress controller version:
NGINX Ingress controller
Release: v1.10.1-hardened1
Build: git-b48c66a2f
Repository: https://github.com/rancher/ingress-nginx
nginx version: nginx/1.25.3
Kubernetes version:
Client Version: v1.30.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.11+rke2r1
Environment:
Cloud provider or hardware configuration: Metallb
OS: Red Hat Enterprise Linux
Kernel (e.g.
uname -a): LinuxRKE2 Version: v1.28.11
Metallb Version: v0.14.5
How was the ingress-nginx-controller installed:
-Below is the second nginx ingress controller passed values
helm -n abcx3apps get values rke2-ingress-nginx-abcx3
Below is User values passed for 1st nginx ingress controller
-Below is the 2nd nginx ingress details:
-kubectl get all -n abcx3apps
-Below is the 1st nginx ingress details:
-kubectl get all -n kube-system
kubectl describe ingressclasses-Current State of the 1st nginx ingress controller:
-Below is description of 1st ingress controller service
kubectl -n <ingresscontrollernamespace> describe svc <ingresscontrollerservicename>-Below is description of 2nd ingress controller service
kubectl -n <ingresscontrollernamespace> describe svc <ingresscontrollerservicename>Below are the logs of 1st ingress controller pods No error logs
Below are the logs of 2nd ingress controller pods No error logs
To reproduce issue
First we will use 2nd ingress controller whose ingressclass is nginx-abcx3 and same deployment yaml and ingress yaml in both scenario
-** deployment yaml**
-** service yaml**
Ingress yaml
Let's apply yamls and see ingress controller logs and deployment status
Ingress controller logs
Lets try to access the application in browser we get below error
Lets see curl output
Now let's keep everything same and just change ingressclass of ingress resource to use ingressclass of 1st igress controller i.e nginx
-Ingress resource of 1st ingress controller
Let's apply ingress yaml and see site in browser
kubectl get ing -n abcx3ns
1st ingress controller logs
And now site is accessible in browser
With ingress resource of 1st ingress controller we are able to access the site in browser but with 2nd ingress controller not a single application is accessible.
lets see curl output of y3.abc.com
curl -iv http://y3.abc.com/sample
Beta Was this translation helpful? Give feedback.
All reactions