Issues with 2024-06 and gson library used by lsp4j v0.22

[scottkurz]

I'm going to write up some details of my problem with 2024-06 here. However, I'll note right at the top here that I'm not sure the latest releases of JDT LS were yet ever intended to target 2024-06. If this is the case, maybe the details aren't so interesting.

DETAILS

In any case, I work with an Eclipse feature (Liberty Tools) using JDT LS, and we are working on a release to support Eclipse 2024-06.

I'm noticing a problem related to the bundle "dependency chain": org.eclipse.jdt.ls.core -> org.eclipse.lsp4j -> com.google.json.

Installing either of the 2024-06 "..for Java developers" or "...Enterprise Java" packages, I notice that com.google.json v2.11.0 gets installed.

When I then install my Liberty Tools feature which pulls in jdtls 1.34.0 and lsp4j 0.22.0 (here's the working PR category.xml) it installs gson 2.10.1 as well.

However, only the lsp4j 0.23.1 that came with the 2024-06 package resolves, the lsp4j 0.22.0 that I brought in for JDT LS does not. I believe the second gson bundle messes up the resolution of lsp4j.

g! lb com.google.gson

   15|Resolved   |    4|Gson (2.11.0)|2.11.0
 1011|Resolved   |    4|Gson (2.10.1)|2.10.1

g! lb org.eclipse.lsp4j
 
  621|Resolved   |    4|org.eclipse.lsp4j (0.23.1.v20240521-1815)|0.23.1.v20240521-1815
 1024|Installed  |    4|org.eclipse.lsp4j (0.22.0.v20240213-2011)|0.22.0.v20240213-2011

g! diag 1024
org.eclipse.lsp4j [1024]
  Unresolved requirement: Import-Package: com.google.gson.annotations; version="[2.9.1,2.11.0)"

I'm not sure why the resolution fails... why can't the gson 2.10.1 bundle resolve this package import? Maybe it has something to do with a "split" package here??

WORKS when org.eclipse.tips.json isn't present

I actually tried the same scenario using the 2024-06 "...for Eclipse Committers" package. Though installing this package originally installed lsp4j v0.23.1 and gson 2.11.0, after installing my Liberty Tools feature, the gson 2.11.0 was uninstalled, gson 2.10.1 was installed and everyone was able to use gson 2.10.1 successfully. So I can resolve both lsp4j bundles to import from a single gson 2.10.1 bundle.

In investigating what the difference was, I believe it might be the org.eclipse.tips.json plugin version 0.3.400, which seems to import gson 2.11.0 version packages.

Import-Package: com.google.gson;version="[2.11.0,3.0.0)"

I'm assuming that the presence of this bundle prevented the Eclipse installer from uninstalling the gson 2.11.0, when using the Java/EnterpriseJava packages, like it uninstalled gson 2.11.0 when using the Committers package?

NEXT STEPS?

I see already merged a PR to move JDT LS to a newer level of LSP4J which makes this whole problem a moot point: #3232

Is there an estimated release date for this next release? If it is soon enough maybe my problem is even less interesting.

Besides all that I'd be interested to learn more about what went wrong along the way, if anyone knows, but if not would be happy to have a way forwards in any case.

Thank you.

[rgrunber]

We tend to release at the end of each month. See https://github.com/eclipse-jdtls/eclipse.jdt.ls/milestones?state=closed for a rough idea of the cadence. We also release snapshot versions of the upcoming release at https://download.eclipse.org/jdtls/snapshots/ .

I'm not sure why there's a resolution error. It should have worked. org.eclipse.lsp4j 0.22.0.v20240213-2011 requires java.package; com.google.gson.annotations [2.9.1,2.11.0) and com.google.gson 2.10.1.v20230109-0753 is clearly providing java.package; java.package:String=com.google.gson.annotations; version:OSGiVersion=2.10.1. There shouldn't be anything wrong with having multiple versions of lsp4j or gson as they are not singleton bundles (they don't contribute directly to the Eclipse platform, just libraries).

Can you explain which split package you're refering to ? I'll probably need to just try the installation myself to get a better answer. Does the problem go away at all if you relaunch the Eclipse instance with the -clean flag ?

[scottkurz]

Thanks @rgrunber for taking a look here...let me respond.

We tend to release at the end of each month.

OK, thank you, (and yes the problem is resolved if we use the newest from master).

Does the problem go away at all if you relaunch the Eclipse instance with the -clean flag ?

No, clean doesn't help.

I'll probably need to just try the installation myself

To recreate:

1. install 2024-06 Enterprise Java (or "for Java Developers" package which doesn't have the OSGi console...note, though, the Committers package does NOT reproduce the error).
2. git clone [email protected]:mezarin/liberty-tools-eclipse.git; cd liberty-tools-eclipse; git checkout b6d9aa2432dc116dfc80523d00f6340308a1f4c7
3. mvn install -DskipTests -DskipITs
4. Install the build output releng/io.openliberty.tools.update/target/io.openliberty.tools.update.eclipse-repository-24.0.6-SNAPSHOT.zip
5. Inspect via OSGI console: lb org.eclipse.lsp4j

Can you explain which split package you're refering to ?

For the 'com.google.gson' bundle, the com.google.gson.annotations package seems to be both imported and exported. I'm just recognizing in general this can complicate things but I'm not saying I've proven this is the problem.

g! bundle 15
...
SymbolicName         com.google.gson
  ...
 Export-Package = com.google.gson;uses:="com.google.gson.reflect,com.google.gson.stream";version="2.11.0",com.google.gson.annotations;version="2.11.0",com.google.gson.reflect;version="2.11.0",com.google.gson.stream;uses:="com.google.gson";version="2.11.0"
 Import-Package = sun.misc;resolution:=optional,com.google.gson.annotations
...
Bundle                  15|Resolved   |    4|com.google.gson (2.11.0)

[rgrunber]

Ok, I can also reproduce your issue. I think this needs to be fixed upstream wherever the gson manifests are injected.

The only missing part is I needed to add https://download.eclipse.org/lsp4j/updates/releases/0.22.0/ to the list of known repositories because it wasn't present in any of the repos that come with the distro by default.

It is completely valid for a package to import its own package(s) that it exports. I think it's meant as a way of saying that the package makes use of that particular package in other parts of the code. I always found it weird to do that but it's technically correct, and by default Eclipse was doing that automatically for a lot of 3rd party libraries, if it detected their usage in the same bundle.

g! p com.google.gson.annotations
osgi.wiring.package; bundle-symbolic-name="com.google.gson"; bundle-version:Version="2.11.0"; version:Version="2.11.0"; osgi.wiring.package="com.google.gson.annotations"<com.google.gson_2.11.0 [15]>
...
...
  com.google.gson_2.10.1 [1007] imports
...
...

The problem, as you've noted, is that both gson bundles import just "com.google.gson.annotations" (implied >= 0.0.0) leaving the door open for resolution between different versions of the same library 😐 gson 2.10.1 should import with a range of [2.10.1,2.11.0) and gson 2.11.0 should import with a range of [2.11.0,2.12.0) to be safe.

com.google.gson_2.10.1.jar
Import-Package = sun.misc;resolution:=optional,com.google.gson.annotations;version="[2.10.1,2.11.0)"

com.google.gson_2.11.0.jar
Import-Package = sun.misc;resolution:=optional,com.google.gson.annotations;version="[2.11.0,2.12.0)"

I manually changed the manifests to have that and re-launched with ./eclipse/eclipse -clean -data foospace to make sure re-resolution occurs. It loaded with nothing in the INSTALLED state.

Update: This is actually an issue with the upstream gson package. Clearly visible at https://github.com/google/gson/blob/9615c8d7c895797872d7e601702e690e5ed7a2e8/gson/pom.xml#L144 . In fact, you can clearly see how we used to package it at https://git.eclipse.org/r/plugins/gitiles/orbit/orbit-recipes/+/refs/heads/master/google/com.google.gson_2.10.1/osgi.bnd before relying purely on upstream's manifest.

[scottkurz]

Thanks for taking a look.

I opened google/gson#2725 to follow-up there.

[chrisrueger]

Just fyi: google/gson#2725 has been fixed. In the next gson release there will not be a (self) Import-Package for com.google.gson.annotations anymore.

[rgrunber]

https://github.com/eclipse-platform/eclipse.platform.releng.aggregator/blob/0a0a26f546c97b153e6312a093baac82e6c527b6/eclipse.platform.releng.prereqs.sdk/eclipse-sdk-prereqs.target#L103-L108 will need to be updated once its released. That's how it'll make its way down to LSP4J and then JDT-LS.

[laeubi]

By the way that's actually a bug in the felix-resolver where giving reexport a very hard time. And as com.google.gson.annotations has no use constraints this can happily give a devastating resolution problem togehter with org.eclipse.lsp4j also having some nasty substituation packages:

• Additional testcase that show some unexpected behavior of resolver eclipse-equinox/equinox#469

[laeubi]

I'm not sure why the resolution fails... why can't the gson 2.10.1 bundle resolve this package import? Maybe it has something to do with a "split" package here??

I just wanted to mention that usually such problems are not caused by the mentioned error but the resolver search space explodes (see here) and then you get strange "unresolved" requirements.

To better understand, latest Equinox has two new commands:

• exportFrameworkState that writes the wiring information into a file that can then be analyzed / debugged
• wires that shows the actual wiring of a bundle what is really helpful in understand what is provided/used and why.