[Question] Unable to build custom JacocoRunner using the script

[tanishiking]

Related #1453

Environment

❯ sysctl machdep.cpu.brand_string
machdep.cpu.brand_string: Apple M2

❯ bazel version
Bazelisk version: development
WARNING: Invoking Bazel in batch mode since it is not invoked from within a workspace (below a directory having a WORKSPACE file).
Build label: 6.1.1
Build target: bazel-out/darwin_arm64-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Wed Mar 15 15:45:56 2023 (1678895156)
Build timestamp: 1678895156
Build timestamp as int: 1678895156

Background

• Code coverage taken from standard Jacoco takes into account the synthetic methods in case classes
  • BTW, why they aren't ignored, although the filters should be applied unconditionally 🤔 https://github.com/jacoco/jacoco/wiki/FilteringOptions
• As commented in build_jacocorunner_bazel script, it seems we need to build the custom Jacoco runner patched in https://github.com/gergelyfabian/jacoco

  • rules_scala/scripts/build_jacocorunner/build_jacocorunner_bazel_5.0+.sh

    Lines 7 to 9 in 6c1f034

    	# 1. Scala support on newer Jacoco versions (including 0.8.6) is still lacking some functionality
    	#
    	# E.g. a lot of generated methods for case classes, lazy vals or other Scala features are causing falsely missed branches in branch coverage.

Problem

I tried to build the custom Jacoco runner with the following patch against current master + tanishiking@9fdb2ba

(Ignoring groovy validation because it fails to build in my local environment, it should be ok because it's test, right?)

when I run ./scripts/build_jacocorunner/build_jacocorunner_bazel_5.0+.sh, it fails with the following error.

ERROR: /private/tmp/bazel_jacocorunner_build/bazel/src/java_tools/junitrunner/java/com/google/testing/coverage/BUILD:33:13: Building src/java_tools/junitrunner/java/com/google/testing/coverage/libJacocoCoverageLib.jar (8 source files) failed: (Exit 1): java failed: error executing command external/remotejdk11_macos_aarch64/bin/java -XX:-CompactStrings '--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED' ... (remaining 17 arguments skipped)
src/java_tools/junitrunner/java/com/google/testing/coverage/ClassProbesMapper.java:32: error: ClassProbesMapper is not abstract and does not override abstract method getInterfaceNames() in IFilterContext
public class ClassProbesMapper extends ClassProbesVisitor implements IFilterContext {
       ^
Target //src/java_tools/junitrunner/java/com/google/testing/coverage:JacocoCoverage_jarjar_deploy.jar failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 4.082s, Critical Path: 3.66s
INFO: 116 processes: 48 internal, 67 darwin-sandbox, 1 worker.
FAILED: Build did NOT complete successfully

This is because

• The patch adds an abstract method getInterfaceNames to IFilterContext gergelyfabian/jacoco@0d63c2c
• Newer version of Bazel (5.0.0~?) tries to implement IFilterContext in ClassProbeMapper
  • bazelbuild/bazel@065e2e8

Questions

• How can we build the custom JacocoRunner with 0.8.7-scala, or is that branch not yet ready?

---

Additional information

0.8.6-scala build also failed

Tried to build with the following patch (because those modules failed to build)

--- a/scripts/build_jacocorunner/build_jacocorunner_bazel_5.0+.sh
+++ b/scripts/build_jacocorunner/build_jacocorunner_bazel_5.0+.sh
@@ -102,7 +102,7 @@ git checkout origin/$jacoco_branch
 for patch in $jacoco_patches; do
   git am $source_path/$patch
 done
-mvn clean install
+mvn clean -pl "!org.jacoco.core.test.validation.kotlin,!org.jacoco.core.test.validation.groovy,!org.jacoco.core.test.validation.scala" install
 )

 (

and failed with the following error

```
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-invoker-plugin:2.0.0:run (test-pom) on project org.jacoco.examples.test: Execution test-pom of goal org.apache.maven.plugins:maven-invoker-plugin:2.0.0:run failed: An API incompatibility was encountered while executing org.apache.maven.plugins:maven-invoker-plugin:2.0.0:run: java.lang.ExceptionInInitializerError: null
[ERROR] -----------------------------------------------------
[ERROR] realm =    plugin>org.apache.maven.plugins:maven-invoker-plugin:2.0.0
[ERROR] strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy
[ERROR] urls[0] = file:/Users/tanishiking/.m2/repository/org/apache/maven/plugins/maven-invoker-plugin/2.0.0/maven-invoker-plugin-2.0.0.jar
[ERROR] urls[1] = file:/Users/tanishiking/.m2/repository/org/apache/maven/shared/maven-invoker/2.2/maven-invoker-2.2.jar
[ERROR] urls[2] = file:/Users/tanishiking/.m2/repository/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar
[ERROR] urls[3] = file:/Users/tanishiking/.m2/repository/org/slf4j/slf4j-jdk14/1.5.6/slf4j-jdk14-1.5.6.jar
[ERROR] urls[4] = file:/Users/tanishiking/.m2/repository/org/slf4j/jcl-over-slf4j/1.5.6/jcl-over-slf4j-1.5.6.jar
[ERROR] urls[5] = file:/Users/tanishiking/.m2/repository/junit/junit/3.8.1/junit-3.8.1.jar
[ERROR] urls[6] = file:/Users/tanishiking/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar
[ERROR] urls[7] = file:/Users/tanishiking/.m2/repository/org/codehaus/plexus/plexus-interactivity-api/1.0-alpha-4/plexus-interactivity-api-1.0-alpha-4.jar
[ERROR] urls[8] = file:/Users/tanishiking/.m2/repository/backport-util-concurrent/backport-util-concurrent/3.1/backport-util-concurrent-3.1.jar
[ERROR] urls[9] = file:/Users/tanishiking/.m2/repository/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.jar
[ERROR] urls[10] = file:/Users/tanishiking/.m2/repository/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.jar
[ERROR] urls[11] = file:/Users/tanishiking/.m2/repository/org/apache/maven/reporting/maven-reporting-api/3.0/maven-reporting-api-3.0.jar
[ERROR] urls[12] = file:/Users/tanishiking/.m2/repository/org/apache/maven/reporting/maven-reporting-impl/2.2/maven-reporting-impl-2.2.jar
[ERROR] urls[13] = file:/Users/tanishiking/.m2/repository/org/apache/maven/doxia/doxia-core/1.2/doxia-core-1.2.jar
[ERROR] urls[14] = file:/Users/tanishiking/.m2/repository/xerces/xercesImpl/2.9.1/xercesImpl-2.9.1.jar
[ERROR] urls[15] = file:/Users/tanishiking/.m2/repository/xml-apis/xml-apis/1.3.04/xml-apis-1.3.04.jar
[ERROR] urls[16] = file:/Users/tanishiking/.m2/repository/commons-lang/commons-lang

/2.4/commons-lang-2.4.jar
[ERROR] urls[17] = file:/Users/tanishiking/.m2/repository/org/apache/httpcomponents
/httpclient/4.0.2/httpclient-4.0.2.jar
[ERROR] urls[18] = file:/Users/tanishiking/.m2/repository/org/apache/httpcomponents
/httpcore/4.0.1/httpcore-4.0.1.jar
[ERROR] urls[19] = file:/Users/tanishiking/.m2/repository/commons-codec/commons-cod
ec/1.3/commons-codec-1.3.jar
[ERROR] urls[20] = file:/Users/tanishiking/.m2/repository/commons-validator/commons
-validator/1.3.1/commons-validator-1.3.1.jar
[ERROR] urls[21] = file:/Users/tanishiking/.m2/repository/commons-beanutils/commons
-beanutils/1.7.0/commons-beanutils-1.7.0.jar
[ERROR] urls[22] = file:/Users/tanishiking/.m2/repository/commons-digester/commons-
digester/1.6/commons-digester-1.6.jar
[ERROR] urls[23] = file:/Users/tanishiking/.m2/repository/commons-logging/commons-l
ogging/1.0.4/commons-logging-1.0.4.jar
[ERROR] urls[24] = file:/Users/tanishiking/.m2/repository/org/apache/maven/shared/m
aven-script-interpreter/1.1/maven-script-interpreter-1.1.jar
[ERROR] urls[25] = file:/Users/tanishiking/.m2/repository/org/codehaus/groovy/groov
y/2.0.1/groovy-2.0.1.jar
[ERROR] urls[26] = file:/Users/tanishiking/.m2/repository/antlr/antlr/2.7.7/antlr-2
.7.7.jar
[ERROR] urls[27] = file:/Users/tanishiking/.m2/repository/org/ow2/asm/asm-tree/4.0/
asm-tree-4.0.jar
[ERROR] urls[28] = file:/Users/tanishiking/.m2/repository/org/ow2/asm/asm-commons/4
.0/asm-commons-4.0.jar
[ERROR] urls[29] = file:/Users/tanishiking/.m2/repository/org/ow2/asm/asm/4.0/asm-4
.0.jar
[ERROR] urls[30] = file:/Users/tanishiking/.m2/repository/org/ow2/asm/asm-util/4.0/
asm-util-4.0.jar
[ERROR] urls[31] = file:/Users/tanishiking/.m2/repository/org/ow2/asm/asm-analysis/
4.0/asm-analysis-4.0.jar
[ERROR] urls[32] = file:/Users/tanishiking/.m2/repository/org/apache/maven/doxia/do
xia-sink-api/1.4/doxia-sink-api-1.4.jar
[ERROR] urls[33] = file:/Users/tanishiking/.m2/repository/org/apache/maven/doxia/do
xia-logging-api/1.4/doxia-logging-api-1.4.jar
[ERROR] urls[34] = file:/Users/tanishiking/.m2/repository/org/apache/maven/doxia/do
xia-site-renderer/1.1.4/doxia-site-renderer-1.1.4.jar
[ERROR] urls[35] = file:/Users/tanishiking/.m2/repository/org/apache/maven/doxia/do
xia-decoration-model/1.1.4/doxia-decoration-model-1.1.4.jar
[ERROR] urls[36] = file:/Users/tanishiking/.m2/repository/org/apache/maven/doxia/do
xia-module-xhtml/1.1.4/doxia-module-xhtml-1.1.4.jar
[ERROR] urls[37] = file:/Users/tanishiking/.m2/repository/org/apache/maven/doxia/do
xia-module-fml/1.1.4/doxia-module-fml-1.1.4.jar
[ERROR] urls[38] = file:/Users/tanishiking/.m2/repository/org/codehaus/plexus/plexu
s-velocity/1.1.7/plexus-velocity-1.1.7.jar
[ERROR] urls[39] = file:/Users/tanishiking/.m2/repository/org/apache/velocity/veloc
ity/1.5/velocity-1.5.jar
[ERROR] urls[40] = file:/Users/tanishiking/.m2/repository/oro/oro/2.0.8/oro-2.0.8.j
ar
[ERROR] urls[41] = file:/Users/tanishiking/.m2/repository/commons-collections/commo
ns-collections/3.2/commons-collections-3.2.jar
[ERROR] urls[42] = file:/Users/tanishiking/.m2/repository/org/beanshell/bsh/2.0b4/b
sh-2.0b4.jar
[ERROR] urls[43] = file:/Users/tanishiking/.m2/repository/org/codehaus/groovy/groov
y-all/2.0.1/groovy-all-2.0.1.jar
[ERROR] urls[44] = file:/Users/tanishiking/.m2/repository/org/apache/ant/ant/1.8.1/
ant-1.8.1.jar
[ERROR] urls[45] = file:/Users/tanishiking/.m2/repository/org/codehaus/plexus/plexu
s-utils/3.0.21/plexus-utils-3.0.21.jar
[ERROR] urls[46] = file:/Users/tanishiking/.m2/repository/commons-io/commons-io/2.2/commons-io-2.2.jar
[ERROR] urls[47] = file:/Users/tanishiking/.m2/repository/org/codehaus/plexus/plexus-interpolation/1.21/plexus-interpolation-1.21.jar
sh-2.0b4.jar
[ERROR] urls[43] = file:/Users/tanishiking/.m2/repository/org/codehaus/groovy/groov
y-all/2.0.1/groovy-all-2.0.1.jar
[ERROR] urls[44] = file:/Users/tanishiking/.m2/repository/org/apache/ant/ant/1.8.1/
ant-1.8.1.jar
[ERROR] urls[45] = file:/Users/tanishiking/.m2/repository/org/codehaus/plexus/plexu
s-utils/3.0.21/plexus-utils-3.0.21.jar
[ERROR] urls[46] = file:/Users/tanishiking/.m2/repository/commons-io/commons-io/2.2
/commons-io-2.2.jar
[ERROR] urls[47] = file:/Users/tanishiking/.m2/repository/org/codehaus/plexus/plexu
sh-2.0b4.jar
[ERROR] urls[43] = file:/Users/tanishiking/.m2/repository/org/codehaus/groovy/groov
y-all/2.0.1/groovy-all-2.0.1.jar
[ERROR] urls[44] = file:/Users/tanishiking/.m2/repository/org/apache/ant/ant/1.8.1/
ant-1.8.1.jar
[ERROR] urls[45] = file:/Users/tanishiking/.m2/repository/org/codehaus/plexus/plexu
s-utils/3.0.21/plexus-utils-3.0.21.jar
[ERROR] urls[46] = file:/Users/tanishiking/.m2/repository/commons-io/commons-io/2.2
/commons-io-2.2.jar
[ERROR] urls[47] = file:/Users/tanishiking/.m2/repository/org/codehaus/plexus/plexu
s-interpolation/1.21/plexus-interpolation-1.21.jar
[ERROR] urls[48] = file:/Users/tanishiking/.m2/repository/org/codehaus/plexus/plexu
s-i18n/1.0-beta-7/plexus-i18n-1.0-beta-7.jar
[ERROR] Number of foreign imports: 1
[ERROR] import: Entry[import from realm ClassRealm[maven.api, parent: null]]
[ERROR]
[ERROR] -----------------------------------------------------
[ERROR] : Unable to make protected void java.lang.Object.finalize() throws java.lang.Throwable accessible: module java.base does not "opens java.lang" to unnamed module @2d917798
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginContainerException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :org.jacoco.examples.test
```

scala branch build pass, but it doesn't work

diff --git a/scripts/build_jacocorunner/build_jacocorunner_bazel_5.0+.sh b/scripts/build_jacocorunner/build_jacocorunner_bazel_5.0+.sh
index 0d1cb0d..fb0b625 100755
--- a/scripts/build_jacocorunner/build_jacocorunner_bazel_5.0+.sh
+++ b/scripts/build_jacocorunner/build_jacocorunner_bazel_5.0+.sh
@@ -57,7 +57,7 @@ jacoco_repo=$build_dir/jacoco
 # Take a fork for Jacoco that contains Scala fixes.
 jacoco_remote=https://github.com/gergelyfabian/jacoco
 # Take further fixes for Scala (2.11, 2.12 and 2.13) - branch in development:
-jacoco_branch=0.8.6-scala
+jacoco_branch=scala

 # Choose the patches that you'd like to use:
 jacoco_patches=""
@@ -68,7 +68,7 @@ jacoco_patches="$jacoco_patches 0001-Build-Jacoco-for-Bazel-5.0+.patch"


 # Jacoco version should be 0.8.6 in any case as Bazel is only compatible with that at this moment.
-jacoco_version=0.8.6
+jacoco_version=0.8.7-SNAPSHOT

 bazel_repo=$build_dir/bazel
 bazel_remote=https://github.com/bazelbuild/bazel
@@ -102,7 +102,7 @@ git checkout origin/$jacoco_branch
 for patch in $jacoco_patches; do
   git am $source_path/$patch
 done
-mvn clean install
+mvn clean -pl "!org.jacoco.core.test.validation.kotlin,!org.jacoco.core.test.validatio
n.groovy,!org.jacoco.core.test.validation.scala" install
 )

 (

In this case, build passes (but why???).
However, when I tested with the custom jacoco runner, in the example repository https://github.com/tanishiking/bazel-playground/tree/main/17-scala-coverage we still have a missing method coverage for synthetic methods by case class.

[liucijus]

@gergelyfabian any chance you can help here?

[gergelyfabian]

Yes, I'm aware of the problems with building for Jacoco 0.8.7. I haven't managed to resolve this yet, also it became lower priority for me as latest Bazel version already uses Jacoco 0.8.7.

I feel, that I'd gladly pass my Jacoco changes to another contributor, but on another note I'm not sure there is readiness from the Jacoco team to work on the Scala fixes (best would be to have them integrated to avoid cases when our fork stops working).

[gergelyfabian]

That's quite shocking, but when using Java 8 (instead of my default Java 17) I managed to build jacoco from the branch...

[gergelyfabian]

export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export PATH=$JAVA_HOME/bin:$PATH

[gergelyfabian]

When using the script with this Java, it builds Jacoco, there is a Bazel error, that I'll try to fix (most probably related to the Bazel version used for building).

[gergelyfabian]

Managed to fix the build, will create a PR with the fixes (including a branch for Bazel itself to add the missing interface).

[gergelyfabian]

@tanishiking Could you please check the solution in #1485?

[gergelyfabian]

Even with my fixes, when I tested the built JacocoRunner, it fails:

Caused by: java.lang.NullPointerException: Cannot invoke "String.endsWith(String)" because the return value of "org.jacoco.core.internal.analysis.filter.IFilterContext.getClassName()" is null
	at org.jacoco.core.internal.analysis.filter.ScalaClassGeneratedMethodsFilter.filter(ScalaClassGeneratedMethodsFilter.java:30)
	at org.jacoco.core.internal.analysis.filter.Filters.filter(Filters.java:65)

Possible that Bazel's Jacoco implementation doesn't work with the Jacoco 0.8.7-scala branch's changes.

[gergelyfabian]

I fixed some null pointer errors on my 0.8.7-scala branch, it should be better now.

[tanishiking]

Thanks a lot @gergelyfabian ! I'll test it out

[gergelyfabian]

This should be fixed after merging #1485.

[tanishiking]

Thanks again @gergelyfabian!