DDF-5324 Upgrades Guava to 25.1 and forces OpenSAML to use the same version #5379

Lambeaux · 2019-09-26T01:24:48Z

What does this PR do?

Upgrades the Guava version used by DDF code and OpenSAML code to 25.1-jre.

Who is reviewing it?

@ryeats
@mojogitoverhere
@brianfelix

Select relevant component teams:

@codice/security

Ask 2 committers to review/merge the PR and tag them here.

@bdeining
@stustison

How should this be tested?

Full build. ✅
SAML Conformance (https://github.com/codice/saml-conformance). ✅
Container starts up, Intrigue is usable logged in. ✅
Features that depend on older Guava versions can be installed (i.e. camel-protobuf). ✅

Any background context you want to provide?

There are several areas of the code where impls are leaking through APIs and, in an OSGi context, are causing unrelated components to inherit problems from their transitive dependencies. This means that the version of Guava that we use and the version of Guava that OpenSAML uses needs to be upgraded in lockstep. The immediate follow-on to this PR will be the following:

Cleaning up the security APIs (Refactor security APIs to not leak dependencies or other implementation details #5327)
Possibly getting off JSR 305 (Migrate off javax annotations and JSR305 #5343)
Looking into catalog features files so component tests use the same building blocks as production (the work to get this upgrade done revealed a lot of discrepancies).

What are the relevant tickets?

Fixes: #5324

Checklist:

Documentation Updated
Update / Add Threat Dragon models
Update / Add Unit Tests
Update / Add Integration Tests

Notes on Review Process

Please see Notes on Review Process for further guidance on requirements for merging and abbreviated reviews.

Review Comment Legend:

✏️ (Pencil) This comment is a nitpick or style suggestion, no action required for approval. This comment should provide a suggestion either as an in line code snippet or a gist.
❓ (Question Mark) This comment is to gain a clearer understanding of design or code choices, clarification is required but action may not be necessary for approval.
❗ (Exclamation Mark) This comment is critical and requires clarification or action before approval.

platform/security/pom.xml

Lambeaux · 2019-09-26T21:31:51Z

Build now.

cxbot · 2019-09-26T21:44:11Z

Internal build has been scheduled, your results will be available at build completion.

ryeats · 2019-09-26T22:37:34Z

platform/security/platform-security-core-api/pom.xml

@@ -225,6 +225,7 @@
                        </Export-Package>
                        <Import-Package>
                            org.opensaml.xmlsec.signature.support.provider,


should we have a version here?

Next chance I get, I'll spin up a container and see what version is being wired up here.

Strange. It's getting wired up correctly without the range.

Import-Package = org.opensaml.xmlsec.signature.support.provider, org.opensaml.core.config;version=3.3.0, ...

Probably because this is the only consumer. I think this will be fine. We're trying to get rid of opensaml anyway.

cxbot · 2019-09-27T03:27:42Z

Refer to this link for build results (access rights to CI server needed):
https://jenkins.codice.org/job/DDF-Jobs/job/pr/job/Linux/7418/

Failed Tests: 0

❌ JOB FAILURE

Lambeaux · 2019-10-01T00:53:31Z

Build now.

cxbot · 2019-10-01T01:06:37Z

Internal build has been scheduled, your results will be available at build completion.

cxbot · 2019-10-01T07:08:01Z

Refer to this link for build results (access rights to CI server needed):
https://jenkins.codice.org/job/DDF-Jobs/job/pr/job/Linux/7449/

Failed Tests: 1

DDF-Jobs/pr/Linux/ddf.catalog.core:catalog-core-directorymonitor: 1

org.codice.ddf.catalog.content.monitor.AsyncFileAlterationObserverTest.testRemovalOfListenerDuringExecution

❌ JOB FAILURE

Lambeaux · 2019-10-01T18:27:41Z

Build now.

cxbot · 2019-10-01T20:58:50Z

Internal build has been scheduled, your results will be available at build completion.

cxbot · 2019-10-01T22:36:02Z

Refer to this link for build results (access rights to CI server needed):
https://jenkins.codice.org/job/DDF-Jobs/job/pr/job/Linux/7460/
❌ JOB FAILURE

mojogitoverhere · 2019-10-02T16:38:12Z

🎉 Hero build was successful! 🎉

Regression tested Intrigue
Passed SAML conformance
camel-protobuf installed without any issues

mojogitoverhere · 2019-10-02T18:13:22Z

build now

cxbot · 2019-10-02T18:21:41Z

Internal build has been scheduled, your results will be available at build completion.

cxbot · 2019-10-02T19:29:04Z

Refer to this link for build results (access rights to CI server needed):
https://jenkins.codice.org/job/DDF-Jobs/job/pr/job/Linux/7466/
❌ JOB FAILURE

Lambeaux · 2019-10-02T21:09:08Z

Build now.

cxbot · 2019-10-02T21:12:48Z

Internal build has been scheduled, your results will be available at build completion.

cxbot · 2019-10-02T22:22:00Z

Refer to this link for build results (access rights to CI server needed):
https://jenkins.codice.org/job/DDF-Jobs/job/pr/job/Linux/7469/
❌ JOB FAILURE

Lambeaux · 2019-10-03T00:50:29Z

Failed to execute goal org.codice.acdebugger:acdebugger-maven-plugin:1.7:start (default) on project test-itests-ddf: Execution default of goal org.codice.acdebugger:acdebugger-maven-plugin:1.7:start failed: Plugin org.codice.acdebugger:acdebugger-maven-plugin:1.7 or one of its dependencies could not be resolved: Could not transfer artifact com.google.errorprone:error_prone_annotations:jar:2.2.0 from/to office (http://nexus.phx.connexta.com:8081/nexus/content/groups/public/): Connect to nexus.phx.connexta.com:8081 [nexus.phx.connexta.com/10.101.1.253] failed: Connection refused (Connection refused) -> [Help 1]
15:21:45 org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codice.acdebugger:acdebugger-maven-plugin:1.7:start (default) on project test-itests-ddf: Execution default of goal org.codice.acdebugger:acdebugger-maven-plugin:1.7:start failed: Plugin org.codice.acdebugger:acdebugger-maven-plugin:1.7 or one of its dependencies could not be resolved: Could not transfer artifact com.google.errorprone:error_prone_annotations:jar:2.2.0 from/to office (http://nexus.phx.connexta.com:8081/nexus/content/groups/public/): Connect to nexus.phx.connexta.com:8081 [nexus.phx.connexta.com/10.101.1.253] failed: Connection refused (Connection refused)

Lambeaux · 2019-10-03T00:50:42Z

Build now.

cxbot · 2019-10-03T00:57:55Z

Internal build has been scheduled, your results will be available at build completion.

cxbot · 2019-10-03T02:49:03Z

Refer to this link for build results (access rights to CI server needed):
https://jenkins.codice.org/job/DDF-Jobs/job/pr/job/Linux/7472/
❌ JOB FAILURE

Lambeaux · 2019-10-03T19:36:23Z

I'm not sure how the poms can be invalid. They are fine locally.

19:49:00 [ERROR] The build could not read 1 project -> [Help 1]
19:49:00 [ERROR]   
19:49:00 [ERROR]   The project ddf.lib.bundles:bundles:[unknown-version] (/jenkins/workspace/DDF-Jobs/pr/Linux/libs/bundles/pom.xml) has 1 error
19:49:00 [ERROR]     Non-resolvable parent POM for ddf.lib.bundles:bundles:[unknown-version]: Could not find artifact ddf.lib:lib:pom:2.19.0-SNAPSHOT and 'parent.relativePath' points at wrong local POM @ line 18, column 13 -> [Help 2]

libs/bundles/libs-bundles-opensaml/pom.xml

bdeining · 2019-10-03T19:37:45Z

I'm not sure how the poms can be invalid. They are fine locally.

19:49:00 [ERROR] The build could not read 1 project -> [Help 1]
19:49:00 [ERROR]   
19:49:00 [ERROR]   The project ddf.lib.bundles:bundles:[unknown-version] (/jenkins/workspace/DDF-Jobs/pr/Linux/libs/bundles/pom.xml) has 1 error
19:49:00 [ERROR]     Non-resolvable parent POM for ddf.lib.bundles:bundles:[unknown-version]: Could not find artifact ddf.lib:lib:pom:2.19.0-SNAPSHOT and 'parent.relativePath' points at wrong local POM @ line 18, column 13 -> [Help 2]

https://github.com/codice/ddf/pull/5379/files#r331216610

Lambeaux · 2019-10-03T19:56:52Z

Build now.

Lambeaux · 2019-10-03T21:29:02Z

Build now.

cxbot · 2019-10-04T02:27:42Z

Internal build has been scheduled, your results will be available at build completion.

cxbot · 2019-10-04T07:01:27Z

Refer to this link for build results (access rights to CI server needed):
https://jenkins.codice.org/job/DDF-Jobs/job/pr/job/Linux/7480/

Build result: FAILURE

[...truncated 14.70 MB...][INFO] BUILD FAILURE[INFO] ------------------------------------------------------------------------[INFO] Total time: 10:24 min[INFO] Finished at: 2019-10-04T07:00:55+00:00[INFO] Final Memory: 427M/2121M[INFO] ------------------------------------------------------------------------[ERROR] Failed to execute goal org.owasp:dependency-check-maven:3.1.1:check (default) on project ddf: One or more exceptions occurred during dependency-check analysis: One or more exceptions occurred during dependency-check analysis[ERROR] The download was interrupted; unable to complete the update[ERROR] -> [Help 1][ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.[ERROR] Re-run Maven using the -X switch to enable full debug logging.[ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles:[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionExceptionBuild step 'Invoke top-level Maven targets' marked build as failureNew run name is 'PR 5379'Taking single-use slave mesos-jenkins-e12506a4-linux-large offline.+ echo ERROR: dockerd exited, or diedArchiving artifactsAdding one-line test results to commit status...Setting status of df2e45a to FAILURE with url https://jenkins.codice.org/job/DDF-Jobs/job/pr/job/Linux/7480/ and message: 'JOB FAILURE: https://jenkins.codice.org/job/DDF-Jobs/job/pr/job/Linux/7480/ 15538 tests run, 86 skipped, 0 failed.'Using context: Internal CI Pipeline
❌ JOB FAILURE

libs/bundles/libs-bundles-opensaml/pom.xml

…ersion

Lambeaux · 2019-10-08T20:59:08Z

Build now.

cxbot · 2019-10-08T21:01:27Z

Internal build has been scheduled, your results will be available at build completion.

cxbot · 2019-10-08T23:01:47Z

Refer to this link for build results (access rights to CI server needed):
https://jenkins.codice.org/job/DDF-Jobs/job/pr/job/Linux/7496/

Build result: FAILURE

[...truncated 5.84 MB...][INFO] --- jacoco-maven-plugin:0.8.2:prepare-agent (default-prepare-agent) @ test ---[INFO] Skipping JaCoCo execution because property jacoco.skip is set.[INFO] jacoco.argline set to empty[INFO] [INFO] --- gmavenplus-plugin:1.5:compile (default) @ test ---[INFO] No sources specified for compilation. Skipping.[INFO] [INFO] --- gmavenplus-plugin:1.5:testCompile (default) @ test ---[INFO] No sources specified for compilation. Skipping.[INFO] [INFO] --- jacoco-maven-plugin:0.8.2:report (default-report) @ test ---[INFO] Skipping JaCoCo execution because property jacoco.skip is set.[INFO] [INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ test ---[INFO] [INFO] --- maven-enforcer-plugin:1.4.1:enforce (check-artifact-size) @ test ---[INFO] Unsupported package type pom. Skipping artifact size enforcement.[INFO] [INFO] --- jacoco-maven-plugin:0.8.2:check (default-check) @ test ---[INFO] Skipping JaCoCo execution because property jacoco.skip is set.[INFO] [INFO] --- maven-install-plugin:2.5.2:install (default-install) @ test ---[INFO] Installing /jenkins/workspace/DDF-Jobs/pr/Linux/distribution/test/pom.xml to /root/.m2/repository/ddf/test/test/2.20.0-SNAPSHOT/test-2.20.0-SNAPSHOT.pom[JENKINS] Archiving disabled
❌ JOB FAILURE

Lambeaux · 2019-10-09T02:07:09Z

Build now.

cxbot · 2019-10-09T02:07:24Z

Internal build has been scheduled, your results will be available at build completion.

cxbot · 2019-10-09T06:49:04Z

Refer to this link for build results (access rights to CI server needed):
https://jenkins.codice.org/job/DDF-Jobs/job/pr/job/Linux/7498/
✅ JOB SUCCESS

Lambeaux requested a review from stustison September 26, 2019 01:24

Lambeaux force-pushed the guava-upgrade-opensaml branch from d2b5ff9 to a0734dc Compare September 26, 2019 02:55

coyotesqrl assigned stustison and bdeining Sep 26, 2019

coyotesqrl requested review from ryeats, bdeining, brianfelix and mojogitoverhere September 26, 2019 15:41

stustison approved these changes Sep 26, 2019

View reviewed changes

Lambeaux force-pushed the guava-upgrade-opensaml branch from a0734dc to c2c647b Compare September 26, 2019 21:27

Lambeaux commented Sep 26, 2019

View reviewed changes

platform/security/pom.xml Show resolved Hide resolved

mojogitoverhere approved these changes Sep 26, 2019

View reviewed changes

ryeats reviewed Sep 26, 2019

View reviewed changes

ryeats approved these changes Sep 26, 2019

View reviewed changes

Lambeaux mentioned this pull request Sep 26, 2019

Upgrade Guava to 25.1 codice/alliance#744

Merged

4 tasks

rymach approved these changes Sep 26, 2019

View reviewed changes

bdeining approved these changes Sep 27, 2019

View reviewed changes

Lambeaux force-pushed the guava-upgrade-opensaml branch from c2c647b to a8f83bb Compare September 30, 2019 23:34

bdeining reviewed Oct 3, 2019

View reviewed changes

libs/bundles/libs-bundles-opensaml/pom.xml Outdated Show resolved Hide resolved

Lambeaux force-pushed the guava-upgrade-opensaml branch from a8f83bb to df2e45a Compare October 3, 2019 19:56

bdeining reviewed Oct 8, 2019

View reviewed changes

libs/bundles/libs-bundles-opensaml/pom.xml Show resolved Hide resolved

DDF-5324 Upgrades Guava to 25.1 and forces opensaml to use the same v…

3f71fb5

…ersion

Lambeaux force-pushed the guava-upgrade-opensaml branch from df2e45a to 3f71fb5 Compare October 8, 2019 20:58

bdeining merged commit d8c2a1a into codice:master Oct 9, 2019

Lambeaux deleted the guava-upgrade-opensaml branch October 9, 2019 21:18

This was referenced Oct 9, 2019

[19.x] DDF-5324 Upgrades Guava to 25.1 and forces opensaml to use the same version #5448

Merged

[18.x] DDF-5324 Upgrades Guava to 25.1 and forces opensaml to use the same version #5453

Merged

Lambeaux mentioned this pull request Oct 5, 2020

DDF-6360 Upgrade to CXF 3.4.0 #6361

Merged

4 tasks

DDF-5324 Upgrades Guava to 25.1 and forces OpenSAML to use the same version #5379

DDF-5324 Upgrades Guava to 25.1 and forces OpenSAML to use the same version #5379

Conversation

Lambeaux commented Sep 26, 2019

What does this PR do?

Who is reviewing it?

Select relevant component teams:

Ask 2 committers to review/merge the PR and tag them here.

How should this be tested?

Any background context you want to provide?

What are the relevant tickets?

Checklist:

Notes on Review Process

Review Comment Legend:

Lambeaux commented Sep 26, 2019

cxbot commented Sep 26, 2019

ryeats Sep 26, 2019

Choose a reason for hiding this comment

Lambeaux Sep 26, 2019

Choose a reason for hiding this comment

Lambeaux Oct 1, 2019 • edited Loading

Choose a reason for hiding this comment

cxbot commented Sep 27, 2019

Failed Tests: 0

Lambeaux commented Oct 1, 2019

cxbot commented Oct 1, 2019

cxbot commented Oct 1, 2019

Failed Tests: 1

DDF-Jobs/pr/Linux/ddf.catalog.core:catalog-core-directorymonitor: 1

Lambeaux commented Oct 1, 2019

cxbot commented Oct 1, 2019

cxbot commented Oct 1, 2019

mojogitoverhere commented Oct 2, 2019

🎉 Hero build was successful! 🎉

mojogitoverhere commented Oct 2, 2019

cxbot commented Oct 2, 2019

cxbot commented Oct 2, 2019

Lambeaux commented Oct 2, 2019

cxbot commented Oct 2, 2019

cxbot commented Oct 2, 2019

Lambeaux commented Oct 3, 2019

Lambeaux commented Oct 3, 2019

cxbot commented Oct 3, 2019

cxbot commented Oct 3, 2019

Lambeaux commented Oct 3, 2019

bdeining commented Oct 3, 2019

Lambeaux commented Oct 3, 2019

Lambeaux commented Oct 3, 2019

cxbot commented Oct 4, 2019

cxbot commented Oct 4, 2019

Build result: FAILURE

Lambeaux commented Oct 8, 2019

cxbot commented Oct 8, 2019

cxbot commented Oct 8, 2019

Build result: FAILURE

Lambeaux commented Oct 9, 2019

cxbot commented Oct 9, 2019

cxbot commented Oct 9, 2019

Lambeaux Oct 1, 2019 •

edited

Loading