ScalaPBModule always recompiles

[nightscape]

I have a Mill build using the ScalaPBModule. Everything is working correctly, but contrary to the way Mill usually efficiently caches results and only does the minimal amount of work, the Protobuf files are recompiled every time I make any changes to Scala code and compile.
Here's a simplified version of my build.mill:

object server extends SbtModule with CommonModule with ScalaPBModule {
  def scalaVersion = scala213
  def scalaPBVersion = "0.11.7"
  def scalaPBSources: T[Seq[PathRef]] = Task.Sources {
    millSourcePath / "src/main/protobuf"
  }
  def scalaPBSearchDeps: Boolean = true

  override def bomIvyDeps = Agg(
    ivy"io.grpc:grpc-bom:1.41.1"
  )
  override def depManagement = Agg(
    ivy"org.apache.spark::spark-sql:$sparkVersion"
  )
  def forkJvmArgs = T { Seq("-Xss32m") }
  val armeriaVersion = "1.9.2"
  val tapirVersion = "1.11.13"
  def ivyDeps = Agg(
    ivy"com.fasterxml.jackson.core:jackson-core:2.15.2",
    ivy"com.fasterxml.jackson.core:jackson-databind:2.15.2",
    ivy"com.fasterxml.jackson.module::jackson-module-scala:2.15.2",
    ivy"com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.15.2",
    ivy"com.google.protobuf:protobuf-java:3.19.1",
    ivy"com.thesamet.scalapb::scalapb-json4s:0.11.1",
  ) ++ Agg(
    ivy"com.softwaremill.sttp.tapir::tapir-core:$tapirVersion",
    ivy"com.softwaremill.sttp.tapir::tapir-json-json4s:$tapirVersion",
    ivy"org.json4s::json4s-native:4.0.7",
    ivy"com.softwaremill.sttp.tapir::tapir-swagger-ui-bundle:$tapirVersion",
    ivy"com.softwaremill.sttp.tapir::tapir-openapi-docs:$tapirVersion",
    ivy"com.softwaremill.sttp.tapir::tapir-files:1.11.12",
    ivy"org.bouncycastle:bcprov-jdk18on:1.76",
    ivy"org.json4s::json4s-jackson:3.7.0-M11",
    ivy"com.linecorp.armeria::armeria-scalapb:$armeriaVersion",
    ...
  ).map(_.excludeOrg(
    "com.fasterxml.jackson.core",
    "com.fasterxml.jackson.module",
    "com.google.protobuf"
  ))
  def prependShellScript = ""
}

and here the relevant output of the Mill run after changing a Scala file:

[build.mill-64/68] compile
[build.mill-64] [info] compiling 1 Scala source to /Users/martin/Workspaces/scala/delta-sharing/out/mill-build/compile.dest/classes ...
[build.mill-64] [info] done compiling
[209/230] server.scalaPBUnpackProto
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / com/linecorp/armeria/grpc/internal_exception_messages.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / scalapb/scalapb.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/geo/type/viewport.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/longrunning/operations.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/type/timeofday.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/type/color.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/type/calendar_period.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/type/interval.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/type/postal_address.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/type/money.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/type/decimal.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/type/datetime.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/type/localized_text.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/type/expr.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/type/quaternion.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/type/latlng.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/type/phone_number.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/type/dayofweek.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/type/month.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/type/fraction.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/type/date.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/rpc/code.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/rpc/error_details.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/rpc/context/attribute_context.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/rpc/status.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/cloud/extended_operations.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/cloud/audit/audit_log.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/usage.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/monitored_resource.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/error_reason.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/auth.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/field_behavior.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/http.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/metric.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/client.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/launch_stage.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/context.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/service.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/log.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/control.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/distribution.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/endpoint.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/system_parameter.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/documentation.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/annotations.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/httpbody.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/consumer.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/source_info.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/billing.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/label.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/monitoring.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/logging.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/visibility.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/resource.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/backend.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/quota.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/api/config_change.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/logging/type/http_request.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/logging/type/log_severity.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/iam/v1/logging/audit_data.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/iam/v1/options.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/iam/v1/iam_policy.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/iam/v1/policy.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/any.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/api.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/compiler/plugin.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/descriptor.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/duration.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/empty.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/field_mask.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/source_context.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/struct.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/timestamp.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/type.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/wrappers.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/any.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/api.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/compiler/plugin.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/descriptor.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/duration.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/empty.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/field_mask.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/source_context.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/struct.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/timestamp.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/type.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / org/apache/hadoop/thirdparty/google/protobuf/wrappers.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / grpc/binlog/v1/binarylog.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / grpc/reflection/v1alpha/reflection.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / grpc/health/v1/health.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / grpc/channelz/v1/channelz.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / grpc/gcp/transport_security_common.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / grpc/gcp/altscontext.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / grpc/gcp/handshaker.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / grpc/lb/v1/load_balancer.proto ...
[209] Warning: Overwriting /Users/martin/Workspaces/scala/delta-sharing/out/server/scalaPBUnpackProto.dest / google/storage/v2/storage.proto ...
[216/230] server.compile
[216] [info] compiling 2 Scala sources to /Users/martin/Workspaces/scala/delta-sharing/out/server/compile.dest/classes ...

I assume this has something to do with

def scalaPBSearchDeps: Boolean = true

because it's referring to my dependencies' *.proto files.

[lefou]

because it's referring to my dependencies' *.proto files.

Did you change the dependencies between the runs? Technically, re-searching the deps should only necessary after a change.

[lefou]

@nightscape Can you look into your out/mill-profile.json. It would be interesting which tasks have cached: false.

[lihaoyi]

@nightscape if you are on the latest version, would also be interesting to look at out/mill-invalidation-tree.json after those invalidations happen so we can see the path through which those tasks are getting invalidated

https://mill-build.org/mill/fundamentals/out-dir.html#_mill_invalidation_tree_json

[nightscape]

It looks like it's something entirely different, nothing to do with ScalaPB:
It seems to be an interference between the Mill BSP service and a different Mill running in the command line.
I started a Mill server in continuous compilation mode

mill -w server.compile

and had VS Code + Metals running.
My repo has a ./mill file which I assume Metals is using.
I have Mill 0.12.5 installed via Homebrew, and my .mill-version also specifies 0.12.5, but it looks like those two still count as different versions when it comes to caching and probably invalidate each others results.
When I run

./mill -w server.compile

everything works smooth and fast as expected.

It's worth noting, that the two Mill processes seem to be aware of each other, at least the command-line Mill issued the following message:

Another Mill process is running 'compiledClassesAndSemanticDbFiles', waiting for it to be done

Maybe it would be worth enhancing that message with a warning if the two Mill versions are not considered identical and invalidate each other's caches.

[lefou]

You're most likely using two different JVMs.