Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dependencies are not resolved when pom used as compile dependency commonly done by gradle published pom.xml #1065

Open
edewit opened this issue Nov 5, 2021 · 12 comments

Comments

@edewit
Copy link

edewit commented Nov 5, 2021

I have the following script
but for some reason jbang doesn't resolve the dependencies:

 edewit@localhost  ~/workspace/minecraft  jbang run --verbose server.java
[jbang] jbang version 0.83.0
[jbang] System Java version detected as 17
[jbang] System Java version matches requested version 17
[jbang] Resolving dependencies...
[jbang]     Resolving com.github.Minestom:Minestom:master-SNAPSHOT...[jbang] Deleting folder /home/edewit/.jbang/cache/jars/server.java.1daf3c6033a38e859e669d8040a68337bdbca312ed2f904937788c9ad8a6eefd.jar.tmp
[jbang] [ERROR] Could not resolve dependency com.github.Minestom:Minestom:master-SNAPSHOT
dev.jbang.cli.ExitException: Could not resolve dependency com.github.Minestom:Minestom:master-SNAPSHOT
	at dev.jbang.dependencies.DependencyUtil.lambda$resolveDependenciesViaAether$3(DependencyUtil.java:201)
	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
	at dev.jbang.dependencies.DependencyUtil.resolveDependenciesViaAether(DependencyUtil.java:208)
	at dev.jbang.dependencies.DependencyUtil.resolveDependencies(DependencyUtil.java:113)
	at dev.jbang.dependencies.DependencyUtil.resolveDependencies(DependencyUtil.java:62)
	at dev.jbang.source.ScriptSource.resolveClassPath(ScriptSource.java:207)
	at dev.jbang.source.RunContext.resolveClassPath(RunContext.java:303)
	at dev.jbang.cli.BaseBuildCommand.buildJar(BaseBuildCommand.java:147)
	at dev.jbang.cli.BaseBuildCommand.build(BaseBuildCommand.java:122)
	at dev.jbang.cli.BaseBuildCommand.buildIfNeeded(BaseBuildCommand.java:86)
	at dev.jbang.cli.Run.prepareArtifacts(Run.java:99)
	at dev.jbang.cli.Run.doCall(Run.java:76)
	at dev.jbang.cli.BaseCommand.call(BaseCommand.java:85)
	at dev.jbang.cli.BaseCommand.call(BaseCommand.java:12)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
	at picocli.CommandLine.access$1300(CommandLine.java:145)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
	at dev.jbang.cli.JBang$3.handle(JBang.java:135)
	at dev.jbang.cli.JBang$3.handle(JBang.java:130)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
	at picocli.CommandLine.execute(CommandLine.java:2078)
	at dev.jbang.Main.main(Main.java:14)
Caused by: org.jboss.shrinkwrap.resolver.api.NoResolvedResultException: Unable to collect/resolve dependency tree for a resolution due to: Could not find artifact org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:jar:3.1.4 in mavencentral (https://repo1.maven.org/maven2/), caused by: Could not find artifact org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:jar:3.1.4 in mavencentral (https://repo1.maven.org/maven2/)
	at org.jboss.shrinkwrap.resolver.impl.maven.MavenWorkingSessionImpl.wrapException(MavenWorkingSessionImpl.java:503)
	at org.jboss.shrinkwrap.resolver.impl.maven.MavenWorkingSessionImpl.resolveDependencies(MavenWorkingSessionImpl.java:242)
	at org.jboss.shrinkwrap.resolver.impl.maven.MavenStrategyStageBaseImpl.using(MavenStrategyStageBaseImpl.java:70)
	at org.jboss.shrinkwrap.resolver.impl.maven.MavenStrategyStageBaseImpl.withTransitivity(MavenStrategyStageBaseImpl.java:52)
	at org.jboss.shrinkwrap.resolver.impl.maven.MavenStrategyStageBaseImpl.withTransitivity(MavenStrategyStageBaseImpl.java:39)
	at dev.jbang.dependencies.DependencyUtil.lambda$resolveDependenciesViaAether$3(DependencyUtil.java:194)
	... 28 more
Caused by: org.eclipse.aether.resolution.DependencyResolutionException: Could not find artifact org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:jar:3.1.4 in mavencentral (https://repo1.maven.org/maven2/)
	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:357)
	at org.jboss.shrinkwrap.resolver.impl.maven.bootstrap.MavenRepositorySystem.resolveDependencies(MavenRepositorySystem.java:121)
	at org.jboss.shrinkwrap.resolver.impl.maven.MavenWorkingSessionImpl.resolveDependencies(MavenWorkingSessionImpl.java:239)
	... 32 more
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not find artifact org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:jar:3.1.4 in mavencentral (https://repo1.maven.org/maven2/)
	at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:424)
	at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:229)
	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:340)
	... 34 more
Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:jar:3.1.4 in mavencentral (https://repo1.maven.org/maven2/)
	at org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(ArtifactTransportListener.java:48)
	at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:369)
	at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:75)
	at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute(BasicRepositoryConnector.java:644)
	at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get(BasicRepositoryConnector.java:262)
	at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:499)
	at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:401)
	... 36 more
[jbang] If you believe this a bug in jbang open issue at https://github.com/jbangdev/jbang/issues

but org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:jar:3.1.4 is definitely in maven central. To be sure I create a quick gradle project with similar dependencies:

repositories {
    // Use JCenter for resolving dependencies.
    jcenter()
    maven { url 'https://repo.spongepowered.org/maven' }
    maven { url 'https://jitpack.io' }
}

dependencies {
    // This dependency is used by the application.
    implementation 'com.github.Minestom:Minestom:master-SNAPSHOT'
}

and that works just fine...

@maxandersen
Copy link
Collaborator

yeah something weird is going on - ill need to debug it.

i can just say i can confirm the issue - best guess is that somehow mavencentral doesn't get properly registered ...but just a hunch.

@maxandersen
Copy link
Collaborator

org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:jar:3.1.4 is a pom - any reason you have that as a compile dependency ?

@maxandersen
Copy link
Collaborator

let me rephrase - somehow "org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:jar:3.1.4" shows up as dependenncy which actually does not exist - "org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:pom:3.1.4" does though.

@maxandersen
Copy link
Collaborator

so I created a minimal pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>gd.wa</groupId>
    <artifactId>minimal-pom</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>minimal-pom</name>
    <url>http://maven.apache.org</url>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <java.version>1.8</java.version>
    </properties>



	<repositories>
		<repository>
		    <id>jitpack.io</id>
		    <url>https://jitpack.io</url>
		</repository>
		<repository>
			<id>sponge</id>
			<url>https://repo.spongepowered.org/maven</url>
			</repository>
		</repositories>



    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.1</version>
                <configuration>
                    <source>${java.version}</source>
                    <target>${java.version}</target>
                </configuration>
            </plugin>
        </plugins>
    </build>

    <dependencies>
        <dependency>
		<groupId>com.github.Minestom</groupId>
		<artifactId>Minestom</artifactId>
		<version>master-SNAPSHOT</version>
	    </dependency>    
    </dependencies>
</project>

and when I do:

mvn dependency:list
I get:

[ERROR] Failed to execute goal on project minimal-pom: Could not resolve dependencies for project gd.wa:minimal-pom:jar:1.0-SNAPSHOT: Could not find artifact org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:jar:3.1.4 in jitpack.io (https://jitpack.io) -> [Help 1]

so jbang is technically reporting correctly here - some metadata in the chain of deps is wrong/bad.

Now I need to figure out why this is not triggered in gradle ...nor when doing mvn compile.

@maxandersen
Copy link
Collaborator

maxandersen commented Nov 6, 2021

The issue is this line: https://github.com/Minestom/DependencyGetter/blob/066247f91b7c90db846134b192fdcf1507855a27/build.gradle.kts#L20

i'm 99% sure that is not valid so when pom.xml data generated it becomes invalid.

`implementation("org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:3.1.4@pom")` could be enough to fix it.

but in general what is happening is a gradle idiom leaking into maven world which is not valid afaics.

@maxandersen
Copy link
Collaborator

so yeah, DependencyGetter generates this pom in https://jitpack.io/com/github/Minestom/DependencyGetter/1.0.1/DependencyGetter-1.0.1.pom

<dependency>
      <groupId>org.jboss.shrinkwrap.resolver</groupId>
      <artifactId>shrinkwrap-resolver-depchain</artifactId>
      <version>3.1.4</version>
      <scope>runtime</scope>
    </dependency>

that dependency does NOT exist.

This does:

<dependency>
      <groupId>org.jboss.shrinkwrap.resolver</groupId>
      <artifactId>shrinkwrap-resolver-depchain</artifactId>
      <version>3.1.4</version>
     <type>pom</type>
      <scope>runtime</scope>
    </dependency>

but that would also be wrong as you would have a pom file on classpath.
it should be something like:

<dependency>
      <groupId>org.jboss.shrinkwrap.resolver</groupId>
      <artifactId>shrinkwrap-resolver-depchain</artifactId>
      <version>${version.shrinkwrap.resolvers}</version>
      <scope>test</scope>
      <type>pom</type>
    </dependency>

which you get if you use the bom as an actual bom - see https://docs.gradle.org/current/userguide/platforms.html#sub:bom_import

@maxandersen maxandersen changed the title Dependencies are not resolved Dependencies are not resolved when pom used as compile dependency Dec 20, 2021
@maxandersen
Copy link
Collaborator

PR opened on minestorm that seem to fix it Minestom/DependencyGetter#4

@linux-china
Copy link
Contributor

How about following style, bom as real Bill of Material and pom for dependencies import?

//DEPS io.quarkus:quarkus-bom:1.11.0.Final@bom
//DEPS org.codehaus.groovy:groovy-all:3.0.9@pom

ShrinkWrap supports to resolve pom as dependencies:

        final ConfigurableMavenResolverSystem resolver = Maven.configureResolver();
        File[] files = resolver.resolve("org.codehaus.groovy:groovy-all:pom:3.0.9").withTransitivity().asFile();
        for (File file : files) {
            System.out.println(file.getAbsolutePath());
        }

@maxandersen
Copy link
Collaborator

It only support one bom. Not multiple which is what needs fixing.

But that doesn't solve this problem as the issue here is in the metadata - the pom.xml published by gradle are wrong/incorrect.

@maxandersen
Copy link
Collaborator

oh i missed your bom / pom differences - interesting idea but separate from this issue which is about pom.xml metadata published on maven central being wrong. you can open that idea as separate but it can't use the qualifier of the maven artifact reference to make the difference as no artifact has @bom artifact type.

@linux-china
Copy link
Contributor

linux-china commented Dec 31, 2021

For example: org.codehaus.groovy:groovy-all:3.0.9@pom could be converted to org.codehaus.groovy:groovy-all:pom:3.0.9 and ShrinkWrap can resolve it.

yes, io.quarkus:quarkus-bom:1.11.0.Final@bom also is converted to io.quarkus:quarkus-bom:pom:1.11.0.Final , and resolved by ShrinkWrap, but it is marked as BOM. For artifacts without version, these artifacts will be sent to PomEquippedResolveStage to get version from effective BOM, and finally all artifacts have versions, and they can be resolved successfully by ShrinkWrap

Maven.resolver().resolve("io.quarkus:quarkus-bom:pom:2.6.1.Final").withoutTransitivity().asFile();
File repositoryHome = new File(System.getProperty("user.home"), ".m2/repository");
File bomFile = new File(repositoryHome, "io/quarkus/quarkus-bom/2.6.1.Final/quarkus-bom-2.6.1.Final.pom");
PomEquippedResolveStage bomResolveStage = Maven.resolver().loadPomFromFile(bomFile).importCompileAndRuntimeDependencies();
MavenResolvedArtifact artifact = bomResolveStage.resolve("com.google.guava:guava").withoutTransitivity().asSingle(MavenResolvedArtifact.class);
assertThat(artifact.getCoordinate().getVersion()).isEqualTo("30.1.1-jre");

@maxandersen
Copy link
Collaborator

Please open other issue for this. We are not going to be able change the semantics for how pom.xml works.

Your idea about allowing jbang users to express in jbang file to fetch dependencies inside a pom we can look at but this issue here is about what to do with the invalid pom.xml files published by gradle. Different issue.

@maxandersen maxandersen changed the title Dependencies are not resolved when pom used as compile dependency Dependencies are not resolved when pom used as compile dependency commonly done by gradle published pom.xml Dec 31, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants