Running the services behind proxy

[nordanz]

Hello all,
I try to install AdvantEDGE on a virtual machine which uses a cntlm proxy, and as a result if the pods in my cluster wants to connect to outside world I have to set the HTTP_PROXY and NO_PROXY env variables explicitly for my pods.

My Questions is, For which Pods I should set these variables?
I know from playing with it, that I have to set it for meep-virt-engine, and now I can see the Dashboard with green light. But when I go to Map view, open street map pod cannot load the tiles (Update: This as well is now fixed by putting the map file and setting the proxy).

And do you have a way to set such default configmaps (I created a configmap for my proxy vars) for all meep deployments?

Thanks in advance.

[nordanz]

In meantime, I tried to run the demo1 and I got this error
{"level":"error","meep.component":"meep-virt-engine","meep.from":"install.go:83","msg":"Failed to install Release [meep-demo1-ue2-svc] at /charts/edge-apps/scenario/demo1/ue2-svc","time":"2022-02-23T11:41:26Z"} {"level":"error","meep.component":"meep-virt-engine","meep.from":"install.go:84","msg":"Error(exit status 1): Error: Deployment.apps \"ue2-svc\" is invalid: spec.template.spec.containers: Required value\n","time":"2022-02-23T11:41:26Z"}

doesn't seem to be related to proxy issue, and still it cannot create ue2-svc as external app. there is no option in external app to set the containers, so I think something else is wrong. it would be great if you provide me with your feedback. thanks.

[nordanz]

Hello @dilallkx,

Thank you and sorry for late response. The run time environment should be fine, I mean it can run and build everything else. the output of kubectl version
Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.2", GitCommit:"092fbfbf53427de67cac1e9fa54aaa09a28371d7", GitTreeState:"clean", BuildDate:"2021-06-16T12:59:11Z", GoVersion:"go1.16.5", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.10", GitCommit:"a7a32748b5c60445c4c7ee904caf01b91f2dbb71", GitTreeState:"clean", BuildDate:"2022-02-16T11:18:16Z", GoVersion:"go1.16.14", Compiler:"gc", Platform:"linux/amd64"}

the output of meepctl version all
Using repo config file: /home/dani/AdvantEDGE/.meepctl-repocfg.yaml Using meepctl config file: /home/dani/.meepctl.yaml { "name": "meepctl", "version": "1.8.1" } { "name": ".meepctl-repocfg.yaml", "version": "1.8.1" } { "name": "meep-cert-manager", "version": "NA" } { "name": "couchdb", "version": "couchdb:3.1.0", "id": "b604d056d8024f10346eab768de7aea06bc0a1b7c55d6087e1b1cd4328c8061c" } { "name": "docker-registry", "version": "registry:2.7.1", "id": "169211e20e2f2d5d115674681eb79d21a217b296b43374b8e39f97fcf866b375" } { "name": "grafana", "version": "grafana/grafana:7.3.5", "id": "511bc20bfcd1b79f3947bb1c33d152f7484e7a91418883fb4dddf71274227321" } { "name": "meep-influxdb", "version": "influxdb:1.8.0-alpine", "id": "5eca9dfe9930a3325323cef801827eb1b0940070465f8f215447b8e732c72b34" } { "name": "meep-ingress", "version": "NA" } { "name": "kube-state-metrics", "version": "quay.io/coreos/kube-state-metrics:v1.9.7", "id": "2f82f0da199c60a7699c43c63a295c44e673242de0b7ee1b17c2d5a23bec34cb" } { "name": "meep-minio", "version": "NA" } { "name": "meep-open-map-tiles", "version": "NA" } { "name": "meep-postgis", "version": "postgis/postgis:12-3.0", "id": "71acda16357f2973034483a4a8363cc9499061120b592bcc3b7f2fbed82da621" } { "name": "meep-prometheus", "version": "NA" } { "name": "meep-redis", "version": "NA" } { "name": "meep-thanos", "version": "NA" } { "name": "meep-thanos-archive", "version": "NA" } { "name": "helm", "version": "v3.3.4", "id": "a61ce5633af99708171414353ed49547cf05013d" } { "name": "docker client", "version": "20.10.12", "id": "e91ed57" } { "name": "docker server", "id": "go1.16.12" } { "name": "k8s client", "version": "v1.21.2", "id": "092fbfbf53427de67cac1e9fa54aaa09a28371d7" } { "name": "k8s server", "version": "v1.21.10", "id": "a7a32748b5c60445c4c7ee904caf01b91f2dbb71" } { "name": "weave", "version": "ghcr.io/weaveworks/launcher/weave-kube:2.8.1", "id": "d797338e7beb17222e10757b71400d8471bdbd9be13b5da38ce2ebf597fb4e63" } { "name": "meep-auth-svc", "version": "meep-docker-registry:30001/meep-auth-svc:latest", "id": "c8f41d8deea6006db78641eab084e88245b1b5e547672e1e0a1d121342962988", "build": "sha256sum:" } { "name": "meep-ingress-certs", "version": "NA" } { "name": "meep-mon-engine", "version": "meep-docker-registry:30001/meep-mon-engine:latest", "id": "37ba8b142cf6b023154d426cf3290744aaa0ce978d9c67705eb4c2c3efa4c077", "build": "sha256sum:" } { "name": "meep-platform-ctrl", "version": "meep-docker-registry:30001/meep-platform-ctrl:latest", "id": "e6462aff9b3be9ec74bba041ec76a5abbec14f086e702600ab0046244b89e425", "build": "sha256sum:" } { "name": "meep-virt-engine", "version": "meep-docker-registry:30001/meep-virt-engine:latest", "id": "efee3734b6e99bccf7fcc39ee48cb7616f5a2fbc9baaec9735be42a775a53796", "build": "8c143ee309648d5b5342d13b745791fe86de78a090132e093066117532a487d1" } { "name": "meep-webhook", "version": "meep-docker-registry:30001/meep-webhook:latest", "id": "33cc4a7798c6984986108ae062f21817424853fcd879c077f05a3e51940df9a8", "build": "869acea124e8c3e8a63c6e2d27dbd51e757821f6deef5eb0b1f546f9681be369" }

and cloned repo hash code is f203f7ab.

Regarding the error, I indeed copied there the logs of meep-virt-engine container. I don't know if the values and deployment content of ue2-svc can help or not but here there are:

# Default values for generic values.yaml
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

deployment:
  enabled: true
  replicaCount: 1
  name: ue2-svc
  apiVersion: v1
  matchLabels:
    meepAppId: b13e0551-e6f2-44bd-b720-6e47380bbf4f

  image:
    name: ue2-svc
    repository:
    pullPolicy: Always
    envvar:
      enabled: false
      env:
    command:
      enabled: false
      command:
      args:

  terminationGracePeriodSeconds: 5

  template:
    labels:
      meepAppId: b13e0551-e6f2-44bd-b720-6e47380bbf4f

  resources:
    requests:
    limits:

  nodeSelector: {}
  tolerations: []
  affinity:
    {}

service:
  enabled: false
  name:
  labels:
  selector:
  type:
  ports:

  meservice:
    enabled: false
    name:
    type: ClusterIP

external:
  enabled: true
  selector:
    meepAppId: b13e0551-e6f2-44bd-b720-6e47380bbf4f
  ingressservicemap:
    - nodePort: 31111
      name: ingress-b13e0551-e6f2-44bd-b720-6e47380bbf4f-31111
      port: 31111
      targetPort: 31111
      protocol: TCP
    - nodePort: 31222
      name: ingress-b13e0551-e6f2-44bd-b720-6e47380bbf4f-31222
      port: 31222
      targetPort: 31222
      protocol: UDP
    - nodePort: 31112
      name: ingress-b13e0551-e6f2-44bd-b720-6e47380bbf4f-31112
      port: 31112
      targetPort: 31112
      protocol: TCP
    - nodePort: 31223
      name: ingress-b13e0551-e6f2-44bd-b720-6e47380bbf4f-31223
      port: 31223
      targetPort: 31223
      protocol: UDP
  egressservicemap:

cat charts/edge-apps/scenario/demo1/ue2-svc/templates/deployment.yaml
{{- if .Values.deployment.enabled}}
apiVersion: apps/{{.Values.deployment.apiVersion}}
kind: Deployment
metadata:
  name: {{template "meep.fullname" .}}
  labels:
    release: {{ .Release.Name }}
spec:
  replicas: {{.Values.deployment.replicaCount}}
  selector:
    matchLabels:
      {{- range $key, $value := .Values.deployment.matchLabels}}
      {{$key}}: {{$value | quote}}
      {{- end}}
  template:
    metadata:
      labels:
        release: {{ .Release.Name }}
        {{- range $key, $value := .Values.deployment.template.labels}}
        {{$key}}: {{$value | quote}}
        {{- end}}

    spec:
      containers:
      {{- if not .Values.external.enabled}}
      {{- with .Values.deployment}}
        - name: {{.image.name}}
        {{- if .image.tag}}
          image: {{.image.repository}}:{{.image.tag}}
        {{else}}
          image: {{.image.repository}}
        {{- end}}
          imagePullPolicy: {{.image.pullPolicy}}
        {{- if .image.command.enabled}}
          command:
          {{- range $cmd := .image.command.command}}
            - {{$cmd}}
          {{- end}}
          args:
          {{- range $arg := .image.command.args}}
            - {{$arg}}
          {{- end}}
        {{- end}}
        {{- if .image.envvar.enabled}}
          env:
          {{- range $key, $val := .image.envvar.env}}
            - name: {{$key}}
              value: {{$val | quote}}
          {{- end}}
        {{- end}}
          resources:
{{toYaml .resources | indent 12}}
      terminationGracePeriodSeconds: {{.terminationGracePeriodSeconds}}
      {{- end}}
      {{- end}}

      nodeSelector:
{{toYaml .Values.deployment.nodeSelector | indent 8}}

      affinity:
{{toYaml .Values.deployment.affinity | indent 8}}

      tolerations:
{{toYaml .Values.deployment.tolerations | indent 8}}
{{- end}}

Thank you in advance.
Ragards!

[dilallkx]

Hi @nordanz,

For external applications defined in a scenario, meep-webhook should inject a sidecar container to manage ingress/egress traffic for the external application. The issue you are observing could be caused by one of the following:

• Helm running in virt-engine is rejecting the chart before the sidecar container gets injected
  • Please provide details about the helm version running in your virt-engine pod
• Webhook is unable to inject the tc-sidecar container
  • Make sure tc-sidecar is enabled in deployment configuration (meepctl-repocfg.yaml)
  • Please provide meep-webhook logs to see if there is an error

Regards!

[nordanz]

Hello Kevin,

In .meep-repoconf.yaml the build, dockerize and deploy flag are set to true for tc-sidecar but still I don't see any logs regarding sidecar in meep-webhook,
{"level":"debug","meep.component":"meep-webhook","meep.from":"webhook.go:88","msg":"RX MSG: Message[SCENARIO-ACTIVATE] Src[edge-apps:meep-sandbox-ctrl] Dst[all:all] Scope[mq:global] Payload[map[sandbox-name:edge-apps scenario-name:demo1]]","time":"2022-03-01T08:09:04Z"} {"level":"debug","meep.component":"meep-webhook","meep.from":"db.go:69","msg":"Redis Connector connecting to meep-redis-master.default.svc.cluster.local:6379","time":"2022-03-01T08:09:04Z"} {"level":"info","meep.component":"meep-webhook","msg":"Redis Connector connected to meep-redis-master.default.svc.cluster.local:6379","time":"2022-03-01T08:09:04Z"} {"level":"info","meep.component":"meep-webhook","msg":"Successfully connected to DB","time":"2022-03-01T08:09:04Z"} {"level":"debug","meep.component":"meep-webhook","meep.from":"model.go:160","msg":"[meep-webhook] Model created edge-apps","time":"2022-03-01T08:09:04Z"}

helm version in meep-virt-engine version.BuildInfo{Version:"v3.3.1", GitCommit:"249e5215cde0c3fa72e27eb7a30e8d55c9696144", GitTreeState:"clean", GoVersion:"go1.14.7"}

Btw, I can run the demo3, But still I don't see a tc-sidecar container. I only have tc-engine pod in sandbox.

Thank you, Regards!

[dilallkx]

Hi @nordanz,

Do you see any "Mutate requests" in your meep-webhook logs?
What is the output of kubectl describe MutatingWebhookConfiguration meep-webhook?

Regards!

[nordanz]

Thanks for your prompt reply,
This is the requested output

Name:         meep-webhook
Namespace:
Labels:       app.kubernetes.io/managed-by=Helm
Annotations:  meta.helm.sh/release-name: meep-webhook
              meta.helm.sh/release-namespace: default
API Version:  admissionregistration.k8s.io/v1
Kind:         MutatingWebhookConfiguration
Metadata:
  Creation Timestamp:  2022-02-23T14:16:54Z
  Generation:          1
  Managed Fields:
    API Version:  admissionregistration.k8s.io/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:meta.helm.sh/release-name:
          f:meta.helm.sh/release-namespace:
        f:labels:
          .:
          f:app.kubernetes.io/managed-by:
      f:webhooks:
        .:
        k:{"name":"meep-webhook.idcc.com"}:
          .:
          f:admissionReviewVersions:
          f:clientConfig:
            .:
            f:caBundle:
            f:service:
              .:
              f:name:
              f:namespace:
              f:path:
              f:port:
          f:failurePolicy:
          f:matchPolicy:
          f:name:
          f:namespaceSelector:
          f:objectSelector:
          f:reinvocationPolicy:
          f:rules:
          f:sideEffects:
          f:timeoutSeconds:
    Manager:         Go-http-client
    Operation:       Update
    Time:            2022-02-23T14:16:54Z
  Resource Version:  216339
  UID:               6588229e-526d-4269-989c-8d25b8387737
Webhooks:
  Admission Review Versions:
    v1beta1
  Client Config:
    Ca Bundle:  LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tC.....
    Service:
      Name:        meep-webhook
      Namespace:   default
      Path:        /mutate
      Port:        443
  Failure Policy:  Ignore
  Match Policy:    Exact
  Name:            meep-webhook.idcc.com
  Namespace Selector:
  Object Selector:
  Reinvocation Policy:  Never
  Rules:
    API Groups:
      apps

    API Versions:
      v1
    Operations:
      CREATE
    Resources:
      deployments
      statefulsets
    Scope:          *
  Side Effects:     Unknown
  Timeout Seconds:  30
Events:             <none>

and there is no "Mutate requests" in meep-webhook.

Thanks again.

[nordanz]

Hello again,

Should I start a new conversation for the problem with demo1? Maybe we need to look at it from different angel :), it's not as well align with topic of discussion.

[dilallkx]

Hi @nordanz,

I agree with moving the demo1 problem to a new discussion (#351).
To confirm, all proxy issues for this discussion are resolved on your end, correct?

Thanks!

[nordanz]

Hello Kevin,

Yes, it seems only the meep-virt-engine needs the proxy configuration for helm chart download. thanks for the support.

[dilallkx]

Perfect, thank you for confirming.