external IP access

[alexfrieden]

Hi folks,
I deployed the outside services:

kubectl -n kafka apply -f outside-services/

However, the result is that there is no external IP:

kubectl -n kafka get all
NAME          READY   STATUS    RESTARTS   AGE
pod/kafka-0   1/1     Running   0          3m
pod/kafka-1   1/1     Running   0          3m
pod/kafka-2   1/1     Running   0          3m
pod/pzoo-0    1/1     Running   0          11m
pod/pzoo-1    1/1     Running   0          11m
pod/pzoo-2    1/1     Running   0          11m
pod/zoo-0     1/1     Running   0          11m
pod/zoo-1     1/1     Running   0          11m

NAME                TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
service/bootstrap   ClusterIP   100.69.215.109   <none>        9092/TCP            3m
service/broker      ClusterIP   None             <none>        9092/TCP            3m
service/outside-0   NodePort    100.69.93.206    <none>        32400:32400/TCP     2m
service/outside-1   NodePort    100.66.39.39     <none>        32401:32401/TCP     2m
service/outside-2   NodePort    100.67.18.146    <none>        32402:32402/TCP     2m
service/pzoo        ClusterIP   None             <none>        2888/TCP,3888/TCP   11m
service/zoo         ClusterIP   None             <none>        2888/TCP,3888/TCP   11m
service/zookeeper   ClusterIP   100.70.47.110    <none>        2181/TCP            11m

NAME                     DESIRED   CURRENT   AGE
statefulset.apps/kafka   3         3         3m
statefulset.apps/pzoo    3         3         11m
statefulset.apps/zoo     2         2         11m

Is this by design? If I want to expose access to this through kafkacat for example, what is the recommendation to do this?

I went through #13 but didn't seem to answer this question (or at least I didn't understand it).

[solsson]

I think https://github.com/Yolean/kubernetes-kafka/tree/master/outside-services#outside-access-with-one-nodeport-per-broker explains why NodePort is default, and where to go from there.

[alexfrieden]

Hi @solsson, thanks for your response. I read through that but shouldn't there be an IP address that is set for external access with NodePort?

Tried testing this out but to no avail:

kafkacat -L -b 100.69.93.206:32400
% ERROR: Failed to acquire metadata: Local: Broker transport failure

[solsson]

What is it you want to do? Expose to outside your cluster but within your network, or to public Internet?

[alexfrieden]

I am actually curious about how to do either. I thought kubectl apply to the outside yamls with the nodePorts would allow public access to it.

[solsson]

It might :) Depending on the networking that surrounds your cluster. But I hope it doesn't. You must look into Kafka authentication before you do that.

[alexfrieden]

@solsson the kops group for this should be pretty open (I'll double check). I am just trying to connect to the cluster to confirm it works which currently it doesn't appear to be exposed. It should have an external IP if the outside broker is apply correct?

[solsson]

Run kubectl -n kafka get pods -l app=kafka -o=jsonpath='{.items[*].metadata.annotations.kafka-listener-outside-host}' to see the IPs of the nodes that brokers happen to be on. I assume based on how GKE works that they're _not_ listed as external IPs in kubectl get nodes -o wide`.

Then from for example a standalone VM instance or from a new cluster, run kafkacat -b [any IP from the annotations]:32400 -L. You'll get a topic list if the machine you're connecting from can access these IPs.

[alexfrieden]

Hmm,
So the results of these seem to fail to connect:

kubectl -n kafka get pods -l app=kafka -o=jsonpath='{.items[*].metadata.annotations.kafka-listener-outside-host}'
172.20.40.143 172.20.55.126 172.20.59.230%

kafkacat -b 172.20.59.230:32400 -L
% ERROR: Failed to acquire metadata: Local: Broker transport failure

Same response for the other two. I have the access to this pretty open so network rules shouldn't be blocking anything.

[solsson]

If you expose NodePort to something else, like a dummy web server pod, does it work with curl from the same test machine?

With kafkacat it sometimes helps to add -d broker to get some more info about the bootstrap process.