Ingress only mode provides incorrect apex

[jpiper]

In ingress-only mode, the apex record cannot be self-referential as

e.g.

k8s_gateway subdomain.example.com {
    resources Ingress
    apex exdns-1-k8s-gateway.kube-system
    secondary exdns-2-k8s-gateway.kube-system
}

exdns-1-k8s-gateway.kube-system.subdomain.example.com won't be a valid record on this NS as this NS is only serving ingress resources.

can I suggest that we make it possible to fully override the apex/secondary values by putting a domain with a . at the end...

k8s_gateway subdomain.example.com {
    resources Ingress
    apex ns1.something.example.com.
    secondary ns2.something.example.com.
}

and then A records can be created that actually point to this ingress-only deployment?

[networkop]

I think this should already be possible.
Will this work https://github.com/ori-edge/k8s_gateway/blob/master/charts/k8s-gateway/values.yaml#L20-L23 ?

[jpiper]

the subdomain gets appended, e.g.

k8s_gateway subdomain.example.com {
    resources Ingress
    apex ns1.example.com.
}

just returns ns1.example.com.subdomain.example.com

[networkop]

ah right, I see, you want it to honour the final dot and not append a subdomain in that case, yeah?

[networkop]

but then where would you create an A record for ns1.something.example.com? DNS server must be aware of this record and be able to resolve it, e.g. dig ns1.something.example.com @ns1.something.example.com should return a valid response.

[jpiper]

In the end I've just manually added some stuff to my corefile which seems to get the job done

hosts domain.example.com {
  ttl 600
  # respond for our own NS1
  <IP_ADDRESS> ns1.domain.example.com
  # We need to hardcode this as the plugin is only watching ingress resources here, otherwise
  # the server won't return a valid A record for it's own apex
  <IP_ADDRESS> gateway-dns.kube-system.domain.example.com
  fallthrough
}
# Respond to NS correctly
template IN NS {
  rcode NOERROR
  answer "domain.example.com. 600 IN NS ns1.domain.example.com."
  additional "ns1.domain.example.com. 600 IN A <IP_ADDRESS>"
}

[networkop]

this is neat! I like solutions that don't involve any code changes.
What is your use case for this? Do you just not want to expose LB services or is it something more complicated?

[jpiper]

In this case, I just wanted to have a separate zone for loadbalancer and ingress resources, e.g. X.lb.example.com and X.ing.example.com

[kainz]

So incidentally I recently also had to do something similar for handling a cluster that was mostly but not entirely accessed via Ingress resources. Also, an override pattern like this would be currently necessary if you want a count of nameservers that is >2.

In that vein, here is an example of what I had to do, leveraging (abusing?) the template's provision for zone plugins.

# Delegated domain
domain: "domain.example"

# Limit what kind of resources to watch, e.g. watchedResources: ["Ingress"]
watchedResources:
  - Ingress

# Enabled fallthrough for k8s_gateway
fallthrough:
  enabled: true
# zones: []

# Optional plugins that will be enabled in the zone, e.g. "forward . /etc/resolve.conf"
extraZonePlugins:
  - name: "hosts domain.example"
    configBlock: |-
      ttl 600
      # respond for our own NS1
      10.42.42.42 ns1.domain.example
      # We need to hardcode this as the plugin is only watching ingress resources here, otherwise
      # the server won't return a valid A record for it's own apex
      10.42.42.42 coredns-external-cluster-k8s-gateway.namespace.domain.example.
      10.42.42.42 noningress-example-host.namespace.domain.example.
      fallthrough
  - name: "template IN SOA domain.example."
    configBlock: |-
      # Respond to NS correctly
      rcode NOERROR
      answer "7201 IN SOA ns1.domain.example. infoemail.domain.example. 9999999999 7200 3600 1209600 60"
      fallthrough
  - name: "template IN NS"
    configBlock: |-
      # Respond to NS correctly
      rcode NOERROR
      answer "domain.example. 600 IN NS ns1.domain.example."
      additional "ns1.domain.example. 600 IN A 10.42.42.42"
  - name: log
  - name: errors
  # Serves a /health endpoint on :8080, required for livenessProbe
  - name: health
    configBlock: |-
      lameduck 5s
  # Serves a /ready endpoint on :8181, required for readinessProbe
  - name: ready
  # Serves a /metrics endpoint on :9153, required for serviceMonitor
  - name: prometheus
    parameters: 0.0.0.0:9153
  # - name: forward
  #  parameters: . /etc/resolv.conf
  # - name: loop
  - name: reload
  - name: loadbalance