Issue requiring modules from a non-parent directory.

[mancusi]

I have a repository which has some interesting quirks including the fact that our main node_modules folder is not in the typical Node module resolution path. Our node_modules is in a directory ./js and our source code is in ./src and a few other directories.

Normally, one could pass a NODE_PATH envvar (see here) which is a list of absolute paths that node_modules could be found at outside of the typical "parent directory" resolution algorithm to run anything in src with node.

I've created a simple example repo which replicates the structure of our repository here. Running bazel build //src:run fails with

node:internal/modules/cjs/loader:936
  throw err;
  ^

Error: Cannot find module 'acorn'
Require stack:
- /private/var/tmp/_bazel_jmancusi/9291d80434b6d660950274f29d1a5907/sandbox/darwin-sandbox/24/execroot/node_modules_not_in_parent_directory/bazel-out/darwin-opt-exec-2B5CBBC6/bin/src/bin.sh.runfiles/node_modules_not_in_parent_directory/src/require_acorn.js
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/private/var/tmp/_bazel_jmancusi/9291d80434b6d660950274f29d1a5907/sandbox/darwin-sandbox/24/execroot/node_modules_not_in_parent_directory/bazel-out/darwin-opt-exec-2B5CBBC6/bin/src/bin.sh.runfiles/node_modules_not_in_parent_directory/src/require_acorn.js:4:16)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/private/var/tmp/_bazel_jmancusi/9291d80434b6d660950274f29d1a5907/sandbox/darwin-sandbox/24/execroot/node_modules_not_in_parent_directory/bazel-out/darwin-opt-exec-2B5CBBC6/bin/src/bin.sh.runfiles/node_modules_not_in_parent_directory/src/require_acorn.js'
  ]
}

As you can see I've been unable to successfully get this working with rules_js. Is there a suggested way of handling repositories of this structure or will I need to move the repository such that the node_modules are linked in a parent directory?

[jbedard]

I think this is difficult because NODE_PATH must be absolute and until runtime we really have no idea where that will be (in the sandbox etc).

You can set it programmatically at runtime using things such as process.env.RUNFILES but it requires a bit of a hack: https://stackoverflow.com/a/33976627

I think ideally you wouldn't depend on this. I assume it is primarily just to hide the node_modules directory? If it's all hidden within bazel is that still necessary?

[gregmagolan]

Yeah. There is no easy way to use NODE_PATH since the directory the build takes place in is a dynamic sandbox under Bazel as @jbedard said. You could in theory hard-code it based on the execroot path and build outside of the sandbox but that is not very portable between machines and probably more trouble than its worth.

In theory you could patch npm_translate_lock to take the pnpm-lock.yaml file from the js folder but still link to the root of the repository under Bazel. That would probably be the easiest way to make your configuration work under rules_js.

It should be just one line of code to change that you can carry as a patch.

rules_js/npm/private/npm_translate_lock.bzl

Line 486 in edd0e66

root_package = rctx.attr.pnpm_lock.package

        # root package is the directory of the pnpm_lock file
        root_package = rctx.attr.pnpm_lock.package

to

        # root package is hard-coded to the root package
        root_package = ""