Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[contrib] add orchestrion integration configuration #3074

Merged
merged 53 commits into from
Jan 20, 2025
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
4631a52
[contrib] add orchestrion integration configuration
RomainMuller Dec 18, 2024
6d487dd
WIP: Set up orchestrion integration testing
RomainMuller Jan 9, 2025
4ac23cc
Add CI viz integration test
RomainMuller Jan 9, 2025
6a90e73
Add GLS integration tests
RomainMuller Jan 10, 2025
ee47a13
Add harness + gqlgen integration test
RomainMuller Jan 10, 2025
d60d93e
Add AWS.v1 tests
RomainMuller Jan 10, 2025
1295f26
Add AWS.v2 tests
RomainMuller Jan 10, 2025
41e9732
Add the rest of integration tests
RomainMuller Jan 10, 2025
882c4c7
Fixup path to requirements-dev.txt
RomainMuller Jan 10, 2025
31fac3e
Add workflow_call trigger to orchestrion workflow
RomainMuller Jan 10, 2025
839cfba
Add option to collect coverage
RomainMuller Jan 10, 2025
435e6ea
Make workflow work when called via workflow_call (event is in foreign…
RomainMuller Jan 10, 2025
03f1102
Quotes around the not
RomainMuller Jan 10, 2025
a674731
Check out orchestrion (go get can't necessarily resolve any SHA githu…
RomainMuller Jan 13, 2025
58d4880
Limit test paralellism in ubuntu
RomainMuller Jan 13, 2025
90908d7
support for DD_TRACE_HTTP_CLIENT_ERROR_STATUSES in orchestrion
eliottness Jan 13, 2025
8eecd63
Model phantom dependency of k8s integration
RomainMuller Jan 13, 2025
c08bdb7
fix comp errors
eliottness Jan 13, 2025
8d5a38e
s/GetEnv/Getenv/
RomainMuller Jan 13, 2025
e0d5d08
Fix linkname directive
RomainMuller Jan 13, 2025
f0b349b
Upgrade to latest dependency
RomainMuller Jan 13, 2025
5b36f94
Move with-context HTTP client short-hand functions to http contrib
RomainMuller Jan 13, 2025
39dd8c8
Also put in replace directive for orchestrion/instrument
RomainMuller Jan 13, 2025
92e9603
Don't use line continuations, PWSH does them differently
RomainMuller Jan 13, 2025
59da424
refactor net/http contrib to remove use of orchestrion/instrument
RomainMuller Jan 14, 2025
1228acc
Automatically detect repo & ref from workflow_ref
RomainMuller Jan 14, 2025
adf5b9e
Merge remote-tracking branch 'origin/main' into romain.marcadier/APPS…
RomainMuller Jan 14, 2025
65d1133
github.workflow_ref is about the top-level workflow, sadly
RomainMuller Jan 14, 2025
2c40d92
Fix breakage on Windows
RomainMuller Jan 14, 2025
efb0ab4
Automatically select go versions to test
RomainMuller Jan 14, 2025
02f0df4
go mod tidy
RomainMuller Jan 14, 2025
e01931b
Merge remote-tracking branch 'origin/main' into romain.marcadier/APPS…
RomainMuller Jan 17, 2025
603e9eb
Update go.mod
RomainMuller Jan 17, 2025
246d470
contrib/net/http: move htto client short-hands into a dedicated clien…
RomainMuller Jan 17, 2025
cca10ca
fix: update integration tests for contribs to skip subpackages
darccio Jan 17, 2025
08cb209
fix(contrib/sirupsen/logrus): remove output assertion
darccio Jan 17, 2025
965bd82
fix(ddtrace/tracer): update TestSpanFromContext to new SpanFromContex…
darccio Jan 17, 2025
db0b477
Add os.OpenFile function to contrib/os
RomainMuller Jan 20, 2025
8128730
Add explanation of what the internal/orchestrion/generator does
RomainMuller Jan 20, 2025
4a44e8a
s/cfg/internalconfig/
RomainMuller Jan 20, 2025
c430ae9
rename _.orchestrion.yml to gls.orchestrion.yml
RomainMuller Jan 20, 2025
afde3ea
Use non-endlessly-respawing orchestrion release
RomainMuller Jan 20, 2025
ada7401
Explicitly go mod download
RomainMuller Jan 20, 2025
c5fc0a1
Try to troubleshoot integration tests failure
RomainMuller Jan 20, 2025
cca226f
Force CGO_ENABLED
RomainMuller Jan 20, 2025
abbb144
Print out complete go env
RomainMuller Jan 20, 2025
b2c8d4c
Remove explicit CGO_ENABLED setting
RomainMuller Jan 20, 2025
9507c38
use go mod tidy instead of go mod download
RomainMuller Jan 20, 2025
beb36c6
Also run linter in integration package directory
RomainMuller Jan 20, 2025
508b9e5
Merge remote-tracking branch 'origin/main' into romain.marcadier/APPS…
RomainMuller Jan 20, 2025
2a8b0d4
Okay got it now :)
RomainMuller Jan 20, 2025
6e09796
Add matrix join point
RomainMuller Jan 20, 2025
0dd7466
Merge branch 'main' into romain.marcadier/APPSEC-55160/orchestrion
RomainMuller Jan 20, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 104 additions & 6 deletions .github/workflows/orchestrion.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
name: Orchestrion
on:
workflow_dispatch: # manually
inputs:
orchestrion-version:
description: Orchestrion version to use for integration testing
type: string
required: false
pull_request:
merge_group:
push:
Expand All @@ -18,9 +23,102 @@ concurrency:
cancel-in-progress: true

jobs:
test:
name: 'Run Tests'
uses: DataDog/orchestrion/.github/workflows/workflow_call.yml@main # we don't want to pin our own action
with:
dd-trace-go-ref: ${{ github.sha }}
runs-on: ubuntu-latest-16-cores
generate:
name: Verify generated files are up-to-date
runs-on: ubuntu-latest
steps:
- name: Checkout Code
uses: actions/checkout@v4
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: stable
cache: true
cache-dependency-path: '**/go.mod'
- name: Run generator
run: go generate ./internal/orchestrion/...
- name: Check for changes
run: git diff --exit-code

integration-test:
strategy:
fail-fast: false
matrix:
runs-on:
- ubuntu
- macos
- windows
go-version: [oldstable, stable]
mode: [DRIVER]
include:
# Alternate build modes (only on ubuntu with oldstable, so we save up CI time)
- runs-on: ubuntu
go-version: oldstable
mode: TOOLEXEC
- runs-on: ubuntu
go-version: oldstable
mode: GOFLAGS
name: Integration Test (${{ matrix.runs-on }} / ${{ matrix.go-version }} / ${{ matrix.mode }})
runs-on: ${{ matrix.runs-on == 'ubuntu' && fromJson('{"labels":"ubuntu-16-core-latest","group":"Large Runner Shared Public"}') || (matrix.runs-on == 'windows' && fromJson('{"labels":"windows-shared-8core","group":"LARGE WINDOWS SHARED"}')) || format('{0}-latest', matrix.runs-on) }}
env:
# Ryuk is problematic with concurrent executions, and unnecessary in ephemeral environments like GHA.
TESTCONTAINERS_RYUK_DISABLED: true
steps:
- name: Checkout Code
uses: actions/checkout@v4
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
cache: true
cache-dependency-path: 'internal/orchestrion/_integration/go.mod'

# ddapm-test-agent is used to observe side effects from the tracer during integration tests.
- name: Set up Python
uses: actions/setup-python@v5

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Code Vulnerability

Workflow depends on a GitHub actions pinned by tag (...read more)

View in Datadog  Leave us feedback  Documentation

with:
python-version: 3.x
cache: pip
cache-dependency-path: 'internal/orchestrion/_integration/internal/agent/requirements-dev.txt'
- name: Install ddapm-test-agent
run: pip install -r internal/orchestrion/_integration/requirements-dev.txt

# If we're in workflow_dispatch, maybe we need to up/downgrade orchestrion
- name: Set up orchestrion
if: github.event_name == 'workflow_dispatch' && github.event.inputs.orchestrion-version != ''
run: |-
go get "github.com/DataDog/orchestrion@${VERSION}"
go mod tidy
working-directory: internal/orchestrion/_integration
env:
VERSION: ${{ github.event.inputs.orchestrion-version }}
# We install the binary to the GOBIN, so it's easy to use
- name: Install orchestrion binary
run: |-
go install "github.com/DataDog/orchestrion"
working-directory: internal/orchestrion/_integration

# Finally, we run the test suite!
- name: Run Tests
shell: bash
run: |-
case "${MODE}" in
"DRIVER")
orchestrion go test -shuffle=on ./...
;;
"TOOLEXEC")
go test -shuffle=on -toolexec='orchestrion toolexec' ./...
;;
"GOFLAGS")
export GOFLAGS="${GOFLAGS} '-toolexec=orchestrion toolexec'"
go test -shuffle=on ./...
;;
*)
echo "Unknown mode: ${MODE}"
;;
esac
working-directory: internal/orchestrion/_integration
env:
GOFLAGS: -tags=githubci${{ matrix.mode == 'DRIVER' && ',buildtag' || ''}} # The "buildtag" tag is used in //dd:span integration tests
MODE: ${{ matrix.mode }}
DD_ORCHESTRION_IS_GOMOD_VERSION: true # Prevent auto-respawn, which is problematic with installs from commit SHA
26 changes: 26 additions & 0 deletions contrib/99designs/gqlgen/orchestrion.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Unless explicitly stated otherwise all files in this repository are licensed
# under the Apache License Version 2.0.
# This product includes software developed at Datadog (https://www.datadoghq.com/).
# Copyright 2023-present Datadog, Inc.
---
# yaml-language-server: $schema=https://datadoghq.dev/orchestrion/schema.json
meta:
name: gopkg.in/DataDog/dd-trace-go.v1/contrib/99designs/gqlgen
description: gqlgen is a Go library for building GraphQL servers without any fuss.

aspects:
- id: New
join-point:
one-of:
- function-call: github.com/99designs/gqlgen/graphql/handler.New
- function-call: github.com/99designs/gqlgen/graphql/handler.NewDefaultServer
advice:
- wrap-expression:
imports:
handler: github.com/99designs/gqlgen/graphql/handler
gqlgentrace: gopkg.in/DataDog/dd-trace-go.v1/contrib/99designs/gqlgen
template: |-
func(s *handler.Server) *handler.Server {
s.Use(gqlgentrace.NewTracer())
return s
}({{ . }})
80 changes: 80 additions & 0 deletions contrib/IBM/sarama.v1/orchestrion.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Unless explicitly stated otherwise all files in this repository are licensed
# under the Apache License Version 2.0.
# This product includes software developed at Datadog (https://www.datadoghq.com/).
# Copyright 2023-present Datadog, Inc.
---
# yaml-language-server: $schema=https://datadoghq.dev/orchestrion/schema.json
meta:
name: gopkg.in/DataDog/dd-trace-go.v1/contrib/IBM/sarama.v1
description: Sarama is a Go library for Apache Kafka

aspects:
- id: NewConsumer
join-point:
one-of:
- function-call: github.com/IBM/sarama.NewConsumer
- function-call: github.com/IBM/sarama.NewConsumerClient
advice:
- wrap-expression:
imports:
saramatrace: gopkg.in/DataDog/dd-trace-go.v1/contrib/IBM/sarama.v1
sarama: github.com/IBM/sarama
template: |-
func(c sarama.Consumer, err error) (sarama.Consumer, error) {
if c != nil {
c = saramatrace.WrapConsumer(c)
}
return c, err
}({{ . }})

- id: NewSyncProducer
join-point:
one-of:
- function-call: github.com/IBM/sarama.NewSyncProducer
- function-call: github.com/IBM/sarama.NewSyncProducerFromClient
advice:
- wrap-expression:
imports:
saramatrace: gopkg.in/DataDog/dd-trace-go.v1/contrib/IBM/sarama.v1
sarama: github.com/IBM/sarama
template: |-
{{- $cfg := .Function.ArgumentOfType "sarama.Config" -}}
func(p sarama.SyncProducer, err error) (sarama.SyncProducer, error) {
if p != nil {
p = saramatrace.WrapSyncProducer(
{{- if $cfg -}}
{{ $cfg }},
{{- else -}}
nil,
{{- end -}}
p,
)
}
return p, err
}({{ . }})

- id: NewAsyncProducer
join-point:
one-of:
- function-call: github.com/IBM/sarama.NewAsyncProducer
- function-call: github.com/IBM/sarama.NewAsyncProducerFromClient
advice:
- wrap-expression:
imports:
saramatrace: gopkg.in/DataDog/dd-trace-go.v1/contrib/IBM/sarama.v1
sarama: github.com/IBM/sarama
template: |-
{{- $cfg := .Function.ArgumentOfType "sarama.Config" -}}
func(p sarama.AsyncProducer, err error) (sarama.AsyncProducer, error) {
if p != nil {
p = saramatrace.WrapAsyncProducer(
{{- if $cfg -}}
{{ $cfg }},
{{- else -}}
nil,
{{- end -}}
p,
)
}
return p, err
}({{ . }})
80 changes: 80 additions & 0 deletions contrib/Shopify/sarama/orchestrion.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Unless explicitly stated otherwise all files in this repository are licensed
# under the Apache License Version 2.0.
# This product includes software developed at Datadog (https://www.datadoghq.com/).
# Copyright 2023-present Datadog, Inc.
---
# yaml-language-server: $schema=https://datadoghq.dev/orchestrion/schema.json
meta:
name: gopkg.in/DataDog/dd-trace-go.v1/contrib/Shopify/sarama
description: Sarama is a Go library for Apache Kafka

aspects:
- id: NewConsumer
join-point:
one-of:
- function-call: github.com/Shopify/sarama.NewConsumer
- function-call: github.com/Shopify/sarama.NewConsumerClient
advice:
- wrap-expression:
imports:
saramatrace: gopkg.in/DataDog/dd-trace-go.v1/contrib/Shopify/sarama
sarama: github.com/Shopify/sarama
template: |-
func(c sarama.Consumer, err error) (sarama.Consumer, error) {
if c != nil {
c = saramatrace.WrapConsumer(c)
}
return c, err
}({{ . }})

- id: NewSyncProducer
join-point:
one-of:
- function-call: github.com/Shopify/sarama.NewSyncProducer
- function-call: github.com/Shopify/sarama.NewSyncProducerFromClient
advice:
- wrap-expression:
imports:
saramatrace: gopkg.in/DataDog/dd-trace-go.v1/contrib/Shopify/sarama
sarama: github.com/Shopify/sarama
template: |-
{{- $cfg := .Function.ArgumentOfType "sarama.Config" -}}
func(p sarama.SyncProducer, err error) (sarama.SyncProducer, error) {
if p != nil {
p = saramatrace.WrapSyncProducer(
{{- if $cfg -}}
{{ $cfg }},
{{- else -}}
nil,
{{- end -}}
p,
)
}
return p, err
}({{ . }})

- id: NewAsyncProducer
join-point:
one-of:
- function-call: github.com/Shopify/sarama.NewAsyncProducer
- function-call: github.com/Shopify/sarama.NewAsyncProducerFromClient
advice:
- wrap-expression:
imports:
saramatrace: gopkg.in/DataDog/dd-trace-go.v1/contrib/Shopify/sarama
sarama: github.com/Shopify/sarama
template: |-
{{- $cfg := .Function.ArgumentOfType "sarama.Config" -}}
func(p sarama.AsyncProducer, err error) (sarama.AsyncProducer, error) {
if p != nil {
p = saramatrace.WrapAsyncProducer(
{{- if $cfg -}}
{{ $cfg }},
{{- else -}}
nil,
{{- end -}}
p,
)
}
return p, err
}({{ . }})
44 changes: 44 additions & 0 deletions contrib/aws/aws-sdk-go-v2/aws/orchestrion.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Unless explicitly stated otherwise all files in this repository are licensed
# under the Apache License Version 2.0.
# This product includes software developed at Datadog (https://www.datadoghq.com/).
# Copyright 2023-present Datadog, Inc.
---
# yaml-language-server: $schema=https://datadoghq.dev/orchestrion/schema.json
meta:
name: gopkg.in/DataDog/dd-trace-go.v1/contrib/aws/aws-sdk-go-v2/aws
description: The official AWS SDK for Go

aspects:
- id: Config
join-point:
struct-literal:
type: github.com/aws/aws-sdk-go-v2/aws.Config
match: value-only
advice:
- wrap-expression:
imports:
awstrace: gopkg.in/DataDog/dd-trace-go.v1/contrib/aws/aws-sdk-go-v2/aws
aws: github.com/aws/aws-sdk-go-v2/aws
template: |-
func(cfg aws.Config) (aws.Config) {
awstrace.AppendMiddleware(&cfg)
return cfg
}({{ . }})

- id: '*Config'
join-point:
one-of:
- struct-literal:
type: github.com/aws/aws-sdk-go-v2/aws.Config
match: pointer-only
- function-call: github.com/aws/aws-sdk-go-v2/aws.NewConfig
advice:
- wrap-expression:
imports:
awstrace: gopkg.in/DataDog/dd-trace-go.v1/contrib/aws/aws-sdk-go-v2/aws
aws: github.com/aws/aws-sdk-go-v2/aws
template: |-
func(cfg *aws.Config) (*aws.Config) {
awstrace.AppendMiddleware(cfg)
return cfg
}({{ . }})
27 changes: 27 additions & 0 deletions contrib/aws/aws-sdk-go/aws/orchestrion.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Unless explicitly stated otherwise all files in this repository are licensed
# under the Apache License Version 2.0.
# This product includes software developed at Datadog (https://www.datadoghq.com/).
# Copyright 2023-present Datadog, Inc.
---
# yaml-language-server: $schema=https://datadoghq.dev/orchestrion/schema.json
meta:
name: gopkg.in/DataDog/dd-trace-go.v1/contrib/aws/aws-sdk-go/aws
description: The official AWS SDK for Go

aspects:
- id: session.NewSession
join-point:
function-call: github.com/aws/aws-sdk-go/aws/session.NewSession
advice:
- wrap-expression:
imports:
awstrace: gopkg.in/DataDog/dd-trace-go.v1/contrib/aws/aws-sdk-go/aws
session: github.com/aws/aws-sdk-go/aws/session
template: |-
func(sess *session.Session, err error) (*session.Session, error) {
if sess != nil {
sess = awstrace.WrapSession(sess)
}
return sess, err
}({{ . }})

Loading
Loading