Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[chore] Create gotest-with-junit Makefile target #11963

Merged
merged 9 commits into from
Dec 23, 2024

Conversation

ankitpatel96
Copy link
Contributor

@ankitpatel96 ankitpatel96 commented Dec 19, 2024

Creates a target to output junit files from testing. This PR also starts to use it in CI - we don't consume the Junits yet but it would be easy to add on. This will be useful for a number of CI/Devx initiatives in the future.

My company, Datadog, plans to use this information to track the build stability of our code in contrib, as well as the stability of code we depend on here in collector.

@ankitpatel96 ankitpatel96 requested a review from a team as a code owner December 19, 2024 22:28
@ankitpatel96 ankitpatel96 changed the title Create gotest-with-junit Makefile target [chore] Create gotest-with-junit Makefile target Dec 19, 2024
Copy link

codecov bot commented Dec 19, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.62%. Comparing base (50104db) to head (c837df7).
Report is 6 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #11963   +/-   ##
=======================================
  Coverage   91.62%   91.62%           
=======================================
  Files         447      447           
  Lines       23731    23739    +8     
=======================================
+ Hits        21743    21751    +8     
  Misses       1613     1613           
  Partials      375      375           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mx-psi mx-psi requested a review from ArthurSens December 20, 2024 11:26
Copy link
Member

@mx-psi mx-psi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ankitpatel96 can you add more on how you plan to use the artifacts?

Also I wonder if these should be uploaded https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/storing-and-sharing-data-from-a-workflow ?

Copy link
Member

@ArthurSens ArthurSens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

We get one JUnit file per go module, it should be possible to correlate files with codeowners :)

@ArthurSens
Copy link
Member

I've added a test to receiver/xreceiver:

func TestFailure(t *testing.T) {
	t.Fail()
}

Then run the new command and got this XML output:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites tests="2" failures="1" errors="0" time="1.835537">
	<testsuite tests="2" failures="1" time="0.660000" name="go.opentelemetry.io/collector/receiver/xreceiver" timestamp="2024-12-20T11:01:25-03:00">
		<properties>
			<property name="go.version" value="go1.23.1 darwin/arm64"></property>
		</properties>
		<testcase classname="go.opentelemetry.io/collector/receiver/xreceiver" name="TestFailure" time="0.000000">
			<failure message="Failed" type="">=== RUN   TestFailure&#xA;--- FAIL: TestFailure (0.00s)&#xA;</failure>
		</testcase>
		<testcase classname="go.opentelemetry.io/collector/receiver/xreceiver" name="TestNewFactoryWithProfiles" time="0.000000"></testcase>
	</testsuite>
</testsuites>

From this file I get good information that I can re-use in automation:

  • Module is present in testsuies[0].testsuite.name
  • Failed tests if testsuites[0].testsuite.testcase[x].failure != nil

Module names can be correlated to the codeowners file so we can ping the relevant people :)

@songy23 songy23 added the ci-cd CI, CD, testing, build issues label Dec 20, 2024
Copy link
Member

@ArthurSens ArthurSens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, actually there's one thing left to do. We probably want to add the test results path to .gitignore

@ankitpatel96
Copy link
Contributor Author

ankitpatel96 commented Dec 20, 2024

@ankitpatel96 can you add more on how you plan to use the artifacts?

Great point! Added a comment about how I / our company plans to use them.

Also I wonder if these should be uploaded docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/storing-and-sharing-data-from-a-workflow ?

Good idea... I will implement that. Even though no one uses these right now the extra time taken should be pretty minimal

@songy23
Copy link
Member

songy23 commented Dec 20, 2024

contrib test failures are due to #11932

@ankitpatel96
Copy link
Contributor Author

Have to update this - the output path was wrong

@ankitpatel96
Copy link
Contributor Author

The artifact is basically a zip with the following list of files:

go.opentelemetry.io-collector-client-junit.xml
go.opentelemetry.io-collector-cmd-builder-junit.xml
go.opentelemetry.io-collector-cmd-mdatagen-junit.xml
go.opentelemetry.io-collector-cmd-otelcorecol-junit.xml
go.opentelemetry.io-collector-component-componentstatus-junit.xml
go.opentelemetry.io-collector-component-componenttest-junit.xml
go.opentelemetry.io-collector-component-junit.xml
go.opentelemetry.io-collector-config-configauth-junit.xml
go.opentelemetry.io-collector-config-configcompression-junit.xml
go.opentelemetry.io-collector-config-configgrpc-junit.xml
go.opentelemetry.io-collector-config-confighttp-junit.xml
go.opentelemetry.io-collector-config-confighttp-xconfighttp-junit.xml
go.opentelemetry.io-collector-config-confignet-junit.xml
go.opentelemetry.io-collector-config-configopaque-junit.xml
go.opentelemetry.io-collector-config-configretry-junit.xml
go.opentelemetry.io-collector-config-configtelemetry-junit.xml
go.opentelemetry.io-collector-config-configtls-junit.xml
go.opentelemetry.io-collector-confmap-internal-e2e-junit.xml
go.opentelemetry.io-collector-confmap-junit.xml
go.opentelemetry.io-collector-confmap-provider-envprovider-junit.xml
go.opentelemetry.io-collector-confmap-provider-fileprovider-junit.xml
go.opentelemetry.io-collector-confmap-provider-httpprovider-junit.xml
go.opentelemetry.io-collector-confmap-provider-httpsprovider-junit.xml
go.opentelemetry.io-collector-confmap-provider-yamlprovider-junit.xml
go.opentelemetry.io-collector-connector-connectortest-junit.xml
go.opentelemetry.io-collector-connector-forwardconnector-junit.xml
go.opentelemetry.io-collector-connector-junit.xml
go.opentelemetry.io-collector-connector-xconnector-junit.xml
go.opentelemetry.io-collector-consumer-consumererror-junit.xml
go.opentelemetry.io-collector-consumer-consumererror-xconsumererror-junit.xml
go.opentelemetry.io-collector-consumer-consumertest-junit.xml
go.opentelemetry.io-collector-consumer-junit.xml
go.opentelemetry.io-collector-consumer-xconsumer-junit.xml
go.opentelemetry.io-collector-exporter-debugexporter-junit.xml
go.opentelemetry.io-collector-exporter-exporterhelper-xexporterhelper-junit.xml
go.opentelemetry.io-collector-exporter-exportertest-junit.xml
go.opentelemetry.io-collector-exporter-junit.xml
go.opentelemetry.io-collector-exporter-nopexporter-junit.xml
go.opentelemetry.io-collector-exporter-otlpexporter-junit.xml
go.opentelemetry.io-collector-exporter-otlphttpexporter-junit.xml
go.opentelemetry.io-collector-exporter-xexporter-junit.xml
go.opentelemetry.io-collector-extension-auth-authtest-junit.xml
go.opentelemetry.io-collector-extension-auth-junit.xml
go.opentelemetry.io-collector-extension-experimental-storage-junit.xml
go.opentelemetry.io-collector-extension-extensioncapabilities-junit.xml
go.opentelemetry.io-collector-extension-extensiontest-junit.xml
go.opentelemetry.io-collector-extension-junit.xml
go.opentelemetry.io-collector-extension-memorylimiterextension-junit.xml
go.opentelemetry.io-collector-extension-zpagesextension-junit.xml
go.opentelemetry.io-collector-featuregate-junit.xml
go.opentelemetry.io-collector-filter-junit.xml
go.opentelemetry.io-collector-internal-e2e-junit.xml
go.opentelemetry.io-collector-internal-fanoutconsumer-junit.xml
go.opentelemetry.io-collector-internal-memorylimiter-junit.xml
go.opentelemetry.io-collector-internal-sharedcomponent-junit.xml
go.opentelemetry.io-collector-internal-tools-junit.xml
go.opentelemetry.io-collector-junit.xml
go.opentelemetry.io-collector-otelcol-junit.xml
go.opentelemetry.io-collector-otelcol-otelcoltest-junit.xml
go.opentelemetry.io-collector-pdata-junit.xml
go.opentelemetry.io-collector-pdata-pprofile-junit.xml
go.opentelemetry.io-collector-pdata-testdata-junit.xml
go.opentelemetry.io-collector-pipeline-junit.xml
go.opentelemetry.io-collector-pipeline-xpipeline-junit.xml
go.opentelemetry.io-collector-processor-batchprocessor-junit.xml
go.opentelemetry.io-collector-processor-junit.xml
go.opentelemetry.io-collector-processor-memorylimiterprocessor-junit.xml
go.opentelemetry.io-collector-processor-processorhelper-xprocessorhelper-junit.xml
go.opentelemetry.io-collector-processor-processortest-junit.xml
go.opentelemetry.io-collector-processor-xprocessor-junit.xml
go.opentelemetry.io-collector-receiver-junit.xml
go.opentelemetry.io-collector-receiver-nopreceiver-junit.xml
go.opentelemetry.io-collector-receiver-otlpreceiver-junit.xml
go.opentelemetry.io-collector-receiver-receivertest-junit.xml
go.opentelemetry.io-collector-receiver-xreceiver-junit.xml
go.opentelemetry.io-collector-scraper-junit.xml
go.opentelemetry.io-collector-semconv-junit.xml
go.opentelemetry.io-collector-service-junit.xml

This is ready to be merged

Copy link
Member

@mx-psi mx-psi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving, I am going to merge this since this would also be useful for open-telemetry/opentelemetry-collector-contrib/issues/36761 which has a lot of support

@mx-psi mx-psi added this pull request to the merge queue Dec 23, 2024
Merged via the queue into open-telemetry:main with commit 84fe00d Dec 23, 2024
38 checks passed
@github-actions github-actions bot added this to the next release milestone Dec 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-cd CI, CD, testing, build issues Skip Contrib Tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants