How to permanently adjust configuration of included services like metrics-server?

[ehlesp]

I have a small k3s cluster which I've installed in a non-default way, and this has provoked an issue with the default configuration of the included metrics-server service. I know what to change in the corresponding Deployment resource (just adding a couple of arguments to its container) but, since all included services are reset to their default manifest after a reboot, I don't know what is the proper way to apply that change and make it stand.

I understand that modifying the manifest found under /var/lib/rancher/k3s/server/manifests would be risky, since the file would be replaced when updating K3s, so it's there any other way to apply permanent changes to the configuration of included services?

In my case the problem is that I've took advantage of the possibility of using different interfaces for external and internal networking, but metrics-server just tries to use the external networking to reach the nodes and I want it to use just the internal networking. Below I paste the config.yaml files of my server node and one agent node for reference.

Server node

cluster-domain: "deimos.cluster.io"
tls-san:
    - "k3sserver01.deimos.cloud"
flannel-backend: host-gw
flannel-iface: "ens19"
bind-address: "0.0.0.0"
https-listen-port: 6443
advertise-address: "10.0.0.1"
advertise-port: 6443
node-ip: "10.0.0.1"
node-external-ip: "200.0.0.5"
node-taint:
    - "k3s-controlplane=true:NoExecute"
log: "/var/log/k3s.log"
disable:
    - servicelb
protect-kernel-defaults: true
secrets-encryption: true
agent-token: "someStrangePassword"

Agent node

flannel-iface: "ens19"
node-ip: "10.0.0.11"
node-external-ip: "200.0.0.11"
server: "https://10.0.0.1:6443"
token: "serverPrefix::node:someStrangePassword"
log: "/var/log/k3s.log"
protect-kernel-defaults: true

[ChristianCiach]

No, there is no other way. The official answer would be to not use the integrated metrics server and deploy it manually.

[ehlesp]

Understood, thanks.

[ChristianCiach]

Just to be clear: This is not an official answer. I am not a maintainer of K3s. But similar questions have been answered this way in the past.

[ehlesp]

Ok, for what I've seen (or, rather, not seen) in the official documentation and googling around, your answer seems correct. There's no option or special folder or anything else in K3s that allows something like patching manifests of included services. Many thanks for your clarification, nevertheless.

[ChristianCiach]

A notable exception are included services that can be customized by deploying HelmChartConfigs. Unfortunately, this currently only applies to the integrated Traefik.

[ehlesp]

Good to know, although currently my experience is strictly limited to the the official Kubernetes toolset: kubectl and kustomize.

On the other hand, I think that, in the same way K3s reads manifests automatically from /var/lib/rancher/k3s/server/manifests, it would be useful if it could also read yaml patches from a path like /etc/rancher/k3s/patches, making easier for anyone wanting to do configuration adjustments to the default configuration of any service, instead of forcing them to do the whole deployment and whatnot for just a small change as in cases like mine. But, well, just my opinion.