Jackson version alignment with Gradle 6

[jjohannes]

Consider publishing Gradle Module Metadata with a platform dependency to the Jackson BOM such that the BOM is automatically used to align versions of multiple Jackson components.

See this mailing list discussion and this blog post for details.

Ideally, all components that use jackson-base as parent should publish Gradle Module Metadata. I created one PR for jackson-base for the shared configuration (FasterXML/jackson-bom#33) and one for jackson-core to show what needs to be done in each POM that has jackson-base as parent (FasterXML/jackson-core#618). Ideally the same change would be done to all these POMs.

Please feel free to do these changes yourself and discard the PRs. I created them to demonstrate how it's done.

[cowtowncoder]

Ok, merged jackson-bom and jackson-core changes. Change to other modules looks simple, although obviously needs some work. The only (?) exceptional case is, I think, jackson-annotations, which does not extend jackson-base.

Will next do jackson-annotations and jackson-databind, to complete core component part.

[cowtowncoder]

Changed jackson-databind as suggested, and realized I have some questions on applicability.

First, for jackson-annotations, things get bit more complicated at least for 3.0, since versioning for annotations will be 3.0, 3.1, and so on, with no patch version.
With 2.x it is still aligned with patch, although patch versions never contain any changes (for a while I thereby had other components refer to .0 of annotations, but users found that confusing so with 2.11 it went back to same version).
So, I wonder what to do with jackson-annotations 3.0: it could depend on jackson-base 3.0.0 (for example), but seems like it might not work well with patch releases.
Alternatively I guess it could just not use same metadata.

My second question has to do with multi-project maven repos like jackson-modules-base.
Can I add the plugin (and comment) in build section of parent only, or does it need to go in actual module poms? I am guessing latter but wanted to double-check; if it went in parent that'd be slightly less work.

[jjohannes]

First, for jackson-annotations, things get bit more complicated at least for 3.0, since versioning for annotations will be 3.0, 3.1, and so on, with no patch version...

This is not a problem. The versions do not all have to be the same. As long as there is always a BOM. If you depend on e.g. jackson-annotations:3.1, it will bring in the 3.1 BOM. And then:

• If you then depend on jackson-core (without version) you will get jackson-core:3.1 (by 3.1 BOM).
• If you depend on jackson-core:3.0, it will be updated to jackson-core:3.1 (by 3.1 BOM).
• If you depend on jackson-core:3.1.1 that will bring in the 3.1.1 BOM which will then be used instead of 3.1 BOM. The 3.1.1 BOM still points at jackson-annotations:3.1 (so the annotations version won't change).

My second question has to do with multi-project maven repos like jackson-modules-base.
Can I add the plugin (and comment) in build section of parent only, or does it need to go in actual module poms? I am guessing latter but wanted to double-check; if it went in parent that'd be slightly less work.

Good question. At the moment it does not work as the plugin would try to publish metadata for the parent as well, which does not work. I am thinking about releasing a new version of the plugin, that does nothing for the parent(s) - projects with packaging=pom. Then you can also consider applying the plugin in jackson-base for all projects. You would still have to add the marker comment to all module POMs though as Gradle does not look into the parents for that. I'll see that I get this update done today.

[jjohannes]

That seems to work well. You can now apply the plugin in a parent (jackson-base if you like - https://github.com/jjohannes/jackson-bom/commit/2a81b79db4398abc3156ce9b3622cefc83fef97c). Then it will be applied to all modules using that parent.

    <plugins>
      ...
      <plugin>
        <groupId>de.jjohannes</groupId>
        <artifactId>gradle-module-metadata-maven-plugin</artifactId>
        <version>0.2.0</version>
        <executions>
          <execution>
            <goals>
              <goal>gmm</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <platformDependencies>
            <dependency>
              <groupId>com.fasterxml.jackson</groupId>
              <artifactId>jackson-bom</artifactId>
              <version>${project.version}</version>
            </dependency>
          </platformDependencies>
        </configuration>
      </plugin>
    </plugins>

The plugin now fails if the marker is missing in a module's pom file. So if you do the above in jackson-base, you can't miss any of the modules as all module builds still missing the marker will now fail, prompting you to add it.

Hope that works for you.

[cowtowncoder]

Ok just to make sure: for 3.0, there will be full versioning for jackson-bom / jackson-base -- 3.0.0, 3.0.1, 3.0.2 -- but only 3.0, 3.1 etc (except for potential unexpected circumstances if 3.0.1 was needed for some reason) of jackson-annotations.

So I guess my question is just on how binding would work for jackson-annotations, specifically; whether its 3.0 version should refer to jackson-bom of 3.0.0; and if so, would this be problematic for following patches of other components.

[jjohannes]

That setup is fine.

whether its 3.0 version should refer to jackson-bom of 3.0.0; and if so, would this be problematic for following patches of other components.

Yes annotations:3.0.0 should refer to jackson-bom:3.0.0 and that is not problematic.

[cowtowncoder]

@jjohannes Ok good, just wanted to clarify.

[cowtowncoder]

@jjohannes Trying to make some more progress here, adding gmm for jackson-annotations (via FasterXML/jackson-annotations#173).

One more question: looking at what plug-in produces, I don't see changes in MANIFEST.MF, but there is new file target/publications/maven/module.json. How will this be used?
Looking at local maven repository directory looks it would become new artifact that ends with ".module" which I guess makes sense, becoming accessible via Maven Central.

[cowtowncoder]

Will also play with possibly auto-enabling plugin via jackson-base, as long it works with multi-module maven-projects; will need to see how that works.

[jjohannes]

Yes. target/publications/maven/module.json becomes the .module file in the repository and that is what Gradle 6+ uses to fetch the additional metadata.

This is the specification of the format:
https://github.com/gradle/gradle/blob/master/subprojects/docs/src/docs/design/gradle-module-metadata-latest-specification.md

And this gives a quick overview:
https://docs.gradle.org/current/userguide/publishing_gradle_module_metadata.html

Let me know if I can be of help with anything.

[cowtowncoder]

Thanks! I should be good, just need to roll out the changes, before 2.12.0-rc1.

[cowtowncoder]

Oh. One thing I realized, about marker: I was wondering why XML processing instruction notation was not used for pom.xml, and instead XML comments are. I don't think that can changed at this point, but while PIs are not widely used, it seems that this is kind of use case that they would fit quite well. I think both kinds of nodes are similarly exposed, although in theory comments are discardable by XML processing tools (allowed to be dropped and not exposed), whereas PIs must be retained (but typically calling apps rarely do anything with them either).

[jjohannes]

That was discussed when we introduced the marker for Gradle about a year ago. It is only there to allow tools that already downloaded the pom metadata to get the information about wether or not the .module file exists without an additional network request. This can be seen as a network performance optimization for large builds that do many requests for metadata files.
See also: https://docs.gradle.org/current/userguide/publishing_gradle_module_metadata.html#sub:interactions-other-build-tools

In the end we decided that the least intrusive way is to use a comment. It is fine if tools that process the pom itself discard it.

[cowtowncoder]

Right inclusion itself makes sense. Thanks for the link!

[cowtowncoder]

Finally found time to add metadata to all official Jackson modules; will be in 2.12.0.

I don't yet have a good way to validate goodness of metadata but am planning adding jackson-integration-tests repo for cross-component compatibility checking and this could perhaps also include tests for this metadata... esp. since there will be fewer restrictions on dependencies that may be added (since it's test-only repo, project(s)).

[cowtowncoder]

Now releasing 2.12.0-rc1 (first pre-release/release candidate). Everything going smooth, except for jackson-jr's "uber-jar" version, which fails: I think a step removes marker XML comment as part of processing, and that makes release fail.
Unfortunately I can not quite reproduce this outside of Maven release plug-in's life-cycle. May have to rework this, unless I can figure out how to disable plugin for that specific component.

[jjohannes]

This is great news. Thanks @cowtowncoder.

Can you share the error you are seeing with jr-all? Maybe I can spot something.

I also now saw that you went ahead with creating https://github.com/FasterXML/jackson-integration-tests. I'll have a look if I can contribute a test in a PR to that repo.

[cowtowncoder]

@jjohannes hi there! I figured out how to invoke the plug-in processing, but went ahead by demoting plug-in invocation to sub-projects away from parent, just to get release done. I am not 100% sure if alignment would make sense for jr-all, given that it is shaded package, including private copy of jackson-core, so that might be fine.

But I think the problem must come from processing done by shade plug-in, which will result in dropping of XML comment; not sure if it might be possible to change lifecycle binding for gmm plug-in to operate on earlier phase, or perhaps allow different marker.
For now, leaving out metadata for jr-all might not be a bad option, but open for improvements.

And yes, if you can think of useful tests for integration-tests package, that would be great!

[jjohannes]

@cowtowncoder I think it makes sense to not add the "platform dependency" to the bom for jr-all. So what you have now for 2.12.0-rc1 looks good. I verified with a test that it works for all other components. This is perfect, thanks!

I created FasterXML/jackson-integration-tests#3 to add the test to your integration test suite. There is only one issue: for some components there are no snapshots available; see comment in PR.

Side note:
Regarding jr-all: If we get the Gradle metadata publishing working, you could make use of capabilities, so that Gradle would fail with a conflict if you have, for example, both jr-all and jackson-core on your classpath.

[cowtowncoder]

@jjohannes it would be good if conflict could be flagged, although it'd be between jackson-jr-objects and jackson-jr-all, not with jackson-core (since jackson-core is shaded so there is no conflict), but classes from jackson-jr-objects (main API of jackson-jr) are left as-is.

[jjohannes]

@cowtowncoder thanks for the background. I might have a look at that and provide a PR if I can make it work.

[cowtowncoder]

So far almost all FasterXML repos are on programme here: one exception is Scala module (due to its use of sbt, mostly): filed an issue:

FasterXML/jackson-module-scala#478

to see if it could be modified to include necessary metadata as well.