ExternalName service with APIService k3s.

[AnirudhPokala]

Environmental Info:
K3s Version: rancher/k3s:v1.25.5-k3s1 (image)
k3s version v1.25.5+k3s1 (48e5d2a)
go version go1.19.4

Cluster Configuration:
vcluster k3s api-server

Describe the bug:
When trying to create an APIService and link with a service of type ExternalName, the apiservice is failing in k3s, but its working with native k8s (tried on kind)

➜  home ✗ k get apiservice
NAME                                   SERVICE                  AVAILABLE                      AGE
v1alpha1.opt.test.io                    default/test-server   False (FailedDiscoveryCheck)   3m5s

On describing I do see this 
Status:
  Conditions:
    Last Transition Time:  2023-06-22T09:40:35Z
    Message:               unsupported service type "ExternalName"
    Reason:                FailedDiscoveryCheck
    Status:                False
    Type:                  Available
Events:                    <none>
---

➜  home ✗ k get svc
NAME                                 TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)    AGE
kubernetes                           ClusterIP      10.12.01.293   <none>           443/TCP    5m56s
test-server                       ExternalName   <none>         test-server   <none>     5m55s

**Steps To Reproduce:**
1. Create an k3s vcluster or any.
2. Create an ExternalName type service and create an APIService linking with it.

**Expected behavior:**
Similar to native k8s. 

[brandond]

K3s sets the apiserver's enable-aggregator-routing flag to true by default, in order to allow the apiserver to tunnel connections to cluster services, as described at https://docs.k3s.io/installation/network-options#control-plane-egress-selector-configuration.

If you want to disable this, and make use of API services outside the cluster, you should start the servers with --egress-selector-mode=agent --kube-apiserver-arg=enable-aggregator-routing=false

[AnirudhPokala]

Cool, thanks @brandond. After configuring the above params, the issue of unsupported service type "ExternalName" went ahead. But now we do see some proxy issues.

  Conditions:
    Last Transition Time:  2023-06-27T12:11:31Z
    Message:               failing or missing response from https://test-server:443/apis/opt.test.io/v1alpha1: Get "https://test-server:443/apis/opt.test.io/v1alpha1": proxy error from 127.0.0.1:6443 while dialing test-server:443, code 503: 503 Service Unavailable
    Reason:                FailedDiscoveryCheck
    Status:                False
    Type:                  Available

We do have the service created already.

[AnirudhPokala]

Also to just debug deployed a sample ubuntu pod and tried to exec into it.. Even it failed with same issue.

/ # k exec -it ubuntu -- bash
Error from server: error dialing backend: proxy error from 127.0.0.1:6443 while dialing 10.96.133.5:10250, code 503: 503 Service Unavailable

[brandond]

You might try using --egress-selector-mode=disabled instead of agent?

Note that you should restart k3s on all nodes in the cluster after changing the egress-selector mode.

[AnirudhPokala]

Thank you able to get it work with specified changes.