Merge branch 'main' of github.com:open-telemetry/opentelemetry-operat…

…or into task/2073-python

.chloggen/1901-support-multi-cont-instr.yaml

@@ -0,0 +1,16 @@
+# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
+change_type: enhancement
+
+# The name of the component, or a single word describing the area of concern, (e.g. operator, target allocator, github action)
+component: operator
+
+# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
+note: "Add support for multi instrumentation"
+
+# One or more tracking issues related to the change
+issues: [1717]
+
+# (Optional) One or more lines of additional information to render under the primary note.
+# These lines will be padded with 2 spaces and then inserted directly into the document.
+# Use pipe (|) for multiline entries.
+subtext:

.chloggen/bump-dotnet-1.0.2.yaml

@@ -0,0 +1,16 @@
+# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
+change_type: enhancement
+
+# The name of the component, or a single word describing the area of concern, (e.g. operator, target allocator, github action)
+component: autoinstrumentation
+
+# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
+note: Bump OpenTelemetry .NET Automatic Instrumentation to 1.0.2
+
+# One or more tracking issues related to the change
+issues: [2168]
+
+# (Optional) One or more lines of additional information to render under the primary note.
+# These lines will be padded with 2 spaces and then inserted directly into the document.
+# Use pipe (|) for multiline entries.
+subtext:

.chloggen/discoverymanager_metrics.yaml

@@ -0,0 +1,16 @@
+# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
+change_type: enhancement
+
+# The name of the component, or a single word describing the area of concern, (e.g. operator, target allocator, github action)
+component: target allocator
+
+# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
+note: Enable discovery manager metrics in target allocator
+
+# One or more tracking issues related to the change
+issues: [2170]
+
+# (Optional) One or more lines of additional information to render under the primary note.
+# These lines will be padded with 2 spaces and then inserted directly into the document.
+# Use pipe (|) for multiline entries.
+subtext:

.chloggen/feat_ta-cliconfig.yaml

@@ -0,0 +1,16 @@
+# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
+change_type: enhancement
+
+# The name of the component, or a single word describing the area of concern, (e.g. operator, target allocator, github action)
+component: target allocator
+
+# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
+note: Allow target allocator to be completely configured via the config file
+
+# One or more tracking issues related to the change
+issues: [2129]
+
+# (Optional) One or more lines of additional information to render under the primary note.
+# These lines will be padded with 2 spaces and then inserted directly into the document.
+# Use pipe (|) for multiline entries.
+subtext:

.chloggen/get-rid-of-autoscaling-v2beta2.yaml

@@ -0,0 +1,20 @@
+# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
+change_type: breaking
+
+# The name of the component, or a single word describing the area of concern, (e.g. operator, target allocator, github action)
+component: operator
+
+# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
+note: Get rid of autoscaling/v2beta2
+
+# One or more tracking issues related to the change
+issues:
+  - 2145
+
+# (Optional) One or more lines of additional information to render under the primary note.
+# These lines will be padded with 2 spaces and then inserted directly into the document.
+# Use pipe (|) for multiline entries.
+subtext: |
+  Kubernetes 1.23 is the minimum available version everywhere after 1.22 deprecation,
+  due to it, the minimum required version has been updated to it, dropping support for
+  autoscaling/v2beta2

.chloggen/proxy.yaml

@@ -0,0 +1,16 @@
+# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
+change_type: enhancement
+
+# The name of the component, or a single word describing the area of concern, (e.g. operator, target allocator, github action)
+component: operator
+
+# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
+note: Propagate proxy environment variables to operands.
+
+# One or more tracking issues related to the change
+issues: [2146]
+
+# (Optional) One or more lines of additional information to render under the primary note.
+# These lines will be padded with 2 spaces and then inserted directly into the document.
+# Use pipe (|) for multiline entries.
+subtext:

.github/dependabot.yml

@@ -66,6 +66,10 @@ updates:
     directory: /
     schedule:
       interval: daily
+    groups:
+      gha-docker:
+        patterns:
+          - "docker/*"
 
   - package-ecosystem: docker
     directory: /

.github/workflows/e2e.yaml

@@ -10,10 +10,6 @@ concurrency:
   group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
   cancel-in-progress: true
 
-env:
-  CURRENT_KIND_VERSION: 0.20.0
-  LEGACY_KIND_VERSION: 0.17.0  # for K8s versions latest kind doesn't support anymore  
-
 jobs:
   e2e-tests:
     name: End-to-end tests
@@ -25,14 +21,15 @@ jobs:
        # All Kubernetes version in between expose the same APIs, hence the operator
        # should be compatible with them.
        kube-version:
-       - "1.19"
+       - "1.23"
        - "1.28"
        group:
          - e2e
          - e2e-instrumentation
          - e2e-upgrade
          - e2e-prometheuscr
          - e2e-autoscale
+         - e2e-multi-instrumentation
 
     steps:
     - name: Set up Go
@@ -42,7 +39,7 @@ jobs:
 
     - name: Setup kind
       env:
-        KIND_VERSION: ${{ matrix.kube_version >= 1.21 && env.KIND_VERSION || env.LEGACY_KIND_VERSION }}
+        KIND_VERSION: "0.20.0"
       run: go install sigs.k8s.io/kind@v${KIND_VERSION}
 
     - name: Check out code into the Go module directory

.github/workflows/publish-autoinstrumentation-e2e-images.yaml

@@ -18,43 +18,13 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  python:
-    runs-on: ubuntu-20.04
-
-    steps:
-      - uses: actions/checkout@v3
-
-      - name: Docker meta
-        id: meta
-        uses: docker/metadata-action@v4
-        with:
-          images: ghcr.io/open-telemetry/opentelemetry-operator/python-test
-
-      - name: Set up QEMU
-        uses: docker/setup-qemu-action@v2
-
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v2
-
-      - name: Cache Docker layers
-        uses: actions/cache@v3
-        with:
-          path: /tmp/.buildx-cache
-          key: ${{ runner.os }}-buildx-${{ github.sha }}
-          restore-keys: |
-            ${{ runner.os }}-buildx-
-      - name: Login to GitHub Package Registry
-        uses: docker/login-action@v2
-        with:
-          registry: ghcr.io
-          username: ${{ github.repository_owner }}
-          password: ${{ secrets.GITHUB_TOKEN }}
-
-      - name: Build and push
-        uses: docker/build-push-action@v4
-        with:
-          context: tests/instrumentation-e2e-apps/python
-          platforms: linux/arm64,linux/amd64,linux/s390x,linux/ppc64le
-          push: ${{ github.event_name == 'push' }}
-          cache-from: type=local,src=/tmp/.buildx-cache
-          cache-to: type=local,dest=/tmp/.buildx-cache
+  golang:
+    uses: ./.github/workflows/reusable-publish-autoinstrumentation-e2e-images.yaml
+    with:
+      language: golang
+      platforms: linux/arm64,linux/amd64,linux/s390x,linux/ppc64le
+  dotnet:
+    uses: ./.github/workflows/reusable-publish-autoinstrumentation-e2e-images.yaml
+    with:
+      language: python
+      platforms: linux/arm64,linux/amd64,linux/s390x,linux/ppc64le

.github/workflows/scorecard.yaml

@@ -10,18 +10,14 @@ concurrency:
   group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
   cancel-in-progress: true
 
-env:
-  CURRENT_KIND_VERSION: 0.20.0
-  LEGACY_KIND_VERSION: 0.17.0  # for K8s versions latest kind doesn't support anymore  
-
 jobs:
   scorecard-tests:
     name: test on k8s
     runs-on: ubuntu-22.04
     strategy:
       matrix:
         kube-version:
-          - "1.19"
+          - "1.23"
           - "1.28"
 
     steps:
@@ -33,7 +29,7 @@ jobs:
 
       - name: Setup kind
         env:
-          KIND_VERSION: ${{ matrix.kube_version >= 1.21 && env.KIND_VERSION || env.LEGACY_KIND_VERSION }}
+          KIND_VERSION: "0.20.0"
         run: go install sigs.k8s.io/kind@v${KIND_VERSION}
 
       - name: Check out code into the Go module directory

Makefile

@@ -207,9 +207,14 @@ e2e-log-operator:
 	kubectl get pod -n opentelemetry-operator-system | grep "opentelemetry-operator" | awk '{print $$1}' | xargs -I {} kubectl logs -n opentelemetry-operator-system {} manager
 	kubectl get deploy -A
 
+# end-to-tests for multi-instrumentation
+.PHONY: e2e-multi-instrumentation
+e2e-multi-instrumentation:
+	$(KUTTL) test --config kuttl-test-multi-instr.yaml
+
 .PHONY: prepare-e2e
 prepare-e2e: kuttl set-image-controller container container-target-allocator container-operator-opamp-bridge start-kind cert-manager install-metrics-server load-image-all deploy
-	TARGETALLOCATOR_IMG=$(TARGETALLOCATOR_IMG) SED_BIN="$(SED)" ./hack/modify-test-images.sh
+	TARGETALLOCATOR_IMG=$(TARGETALLOCATOR_IMG) OPERATOR_IMG=$(IMG) SED_BIN="$(SED)" ./hack/modify-test-images.sh
 
 .PHONY: enable-prometheus-feature-flag
 enable-prometheus-feature-flag:

README.md

@@ -302,7 +302,7 @@ The possible values for the annotation can be
 * `"my-other-namespace/my-instrumentation"` - name and namespace of `Instrumentation` CR instance in another namespace.
 * `"false"` - do not inject
 
-#### Multi-container pods
+#### Multi-container pods with single instrumentation
 
 If nothing else is specified, instrumentation is performed on the first container available in the pod spec.
 In some cases (for example in the case of the injection of an Istio sidecar) it becomes necessary to specify on which container(s) this injection must be performed.
@@ -342,6 +342,90 @@ In the above case, `myapp` and `myapp2` containers will be instrumented, `myapp3
 
 > 🚨 **NOTE**: Go auto-instrumentation **does not** support multicontainer pods. When injecting Go auto-instrumentation the first pod should be the only pod you want instrumented.
 
+#### Multi-container pods with multiple instrumentations
+
+Works only when `operator.autoinstrumentation.multi-instrumentation` feature is `enabled`.
+
+Annotations defining which language instrumentation will be injected are required. When feature is enabled, specific for Instrumentation language containers annotations are used:
+
+Java:
+```bash
+instrumentation.opentelemetry.io/java-container-names: "java1,java2"
+```
+
+NodeJS:
+```bash
+instrumentation.opentelemetry.io/nodejs-container-names: "nodejs1,nodejs2"
+```
+
+Python:
+```bash
+instrumentation.opentelemetry.io/python-container-names: "python1,python3"
+```
+
+DotNet:
+```bash
+instrumentation.opentelemetry.io/dotnet-container-names: "dotnet1,dotnet2"
+```
+
+Go:
+```bash
+instrumentation.opentelemetry.io/go-container-names: "go1"
+```
+
+ApacheHttpD:
+```bash
+instrumentation.opentelemetry.io/apache-httpd-container-names: "apache1,apache2"
+```
+
+SDK:
+```bash
+instrumentation.opentelemetry.io/sdk-container-names: "app1,app2"
+```
+
+If language instrumentation specific container names are not specified, instrumentation is performed on the first container available in the pod spec (only if single instrumentation injection is configured).
+
+In some cases containers in the pod are using different technologies. It becomes necessary to specify language instrumentation for container(s) on which this injection must be performed.
+
+For this, we will use language instrumentation specific container names annotation for which we will indicate one or more container names (`.spec.containers.name`) on which the injection must be made:
+
+```yaml
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: my-deployment-with-multi-containers-multi-instrumentations
+spec:
+  selector:
+    matchLabels:
+      app: my-pod-with-multi-containers-multi-instrumentations
+  replicas: 1
+  template:
+    metadata:
+      labels:
+        app: my-pod-with-multi-containers-multi-instrumentations
+      annotations:
+        instrumentation.opentelemetry.io/inject-java: "true"
+        instrumentation.opentelemetry.io/java-container-names: "myapp,myapp2"
+        instrumentation.opentelemetry.io/inject-python: "true"
+        instrumentation.opentelemetry.io/python-container-names: "myapp3"
+    spec:
+      containers:
+      - name: myapp
+        image: myImage1
+      - name: myapp2
+        image: myImage2
+      - name: myapp3
+        image: myImage3
+```
+
+In the above case, `myapp` and `myapp2` containers will be instrumented using Java and `myapp3` using Python instrumentation.
+
+**NOTE**: Go auto-instrumentation **does not** support multicontainer pods. When injecting Go auto-instrumentation the first container should be the only you want to instrument.
+
+**NOTE**: This type of instrumentation **does not** allow to instrument a container with multiple language instrumentations.
+
+**NOTE**: `instrumentation.opentelemetry.io/container-names` annotation is not used for this feature.
+
 #### Use customized or vendor instrumentation
 
 By default, the operator uses upstream auto-instrumentation libraries. Custom auto-instrumentation can be configured by
@@ -403,8 +487,8 @@ instrumentation.opentelemetry.io/inject-sdk: "true"
 The operator allows specifying, via the feature gates,  which languages the Instrumentation resource may instrument.
 These feature gates must be passed to the operator via the `--feature-gates` flag.
 The flag allows for a comma-delimited list of feature gate identifiers.
-Prefix a gate with '-' to disable support for the corresponding language.
-Prefixing a gate with '+' or no prefix will enable support for the corresponding language.
+Prefix a gate with '-' to disable support for the corresponding language or multi instrumentation feature.
+Prefixing a gate with '+' or no prefix will enable support for the corresponding language or multi instrumentation feature.
 If a language is enabled by default its gate only needs to be supplied when disabling the gate.
 
 | Language      | Gate                                        | Default Value |
@@ -418,6 +502,10 @@ If a language is enabled by default its gate only needs to be supplied when disa
 
 Language not specified in the table are always supported and cannot be disabled.
 
+OpenTelemetry Operator allows to instrument multiple containers using multiple language specific instrumentations.
+These feature can be enabled using `operator.autoinstrumentation.multi-instrumentation` flag. By default flag is `disabled`.
+For more information about multi-instrumentation feature capabilities please see [Multi-container pods with multiple instrumentations](#Multi-container-pods-with-multiple-instrumentations).
+
 ### Target Allocator
 
 The OpenTelemetry Operator comes with an optional component, the [Target Allocator](/cmd/otel-allocator/README.md) (TA). When creating an OpenTelemetryCollector Custom Resource (CR) and setting the TA as enabled, the Operator will create a new deployment and service to serve specific `http_sd_config` directives for each Collector pod as part of that CR. It will also change the Prometheus receiver configuration in the CR, so that it uses the [http_sd_config](https://prometheus.io/docs/prometheus/latest/http_sd/) from the TA. The following example shows how to get started with the Target Allocator:

autoinstrumentation/dotnet/version.txt

@@ -1 +1 @@
-1.0.0
+1.0.2