fix: cache local environment values

[Tofandel]

Yes this is a bit dirty because the report is now generated once at the beginning of a command which could be considered a side effect (but normally treeshaking will only include it only if needed), but the whole process.report is a weirdly built API meant for debugging and not really meant to be used in a normal program

But somehow when the call is done in the beginning of the process, this call is very fast, but when it's called after having run a http request (as is currently the case, a single call takes a lot more time to complete, it takes 40s or more on my system when called at the end of npm upgrade) but only a few milliseconds when called at the beginning (this is why it's best to run it outside the function at the beginning of the process as a side effect instead of calling getReport on demand and cache the result)

Here is a log of console.time('report') and console.timeEnd('report') before and after the getReport call
when run in the end of the upgrade command:
⠼report: 3:10.573 (m:ss.mmm)
when run in the beginning of the process at the top level
report: 1.943ms

This fixes npm hanging npm/cli#4028, npm/cli#7814, npm/cli#7868

[Tofandel]

Another option is to copy the logic from yarn which also fixed this issue by looking into the /usr/bin/ldd file and fallback to getReport after

https://github.com/yarnpkg/berry/blob/f59bbf9f3828865c14b06a3e5cc3ae284a0db78d/packages/yarnpkg-core/sources/nodeUtils.ts#L28-L67

[IngFu]

Another option is to copy the logic from yarn which also fixed this issue by looking into the /usr/bin/ldd file and fallback to getReport after

https://github.com/yarnpkg/berry/blob/f59bbf9f3828865c14b06a3e5cc3ae284a0db78d/packages/yarnpkg-core/sources/nodeUtils.ts#L28-L67

Is getReport() called once during npm install or multiple times?

[Tofandel]

Currently multiple times because it's called at every checkPlatform, which is done in a loop once per installed package during npm install, which is what this PR is attempting to fix. I have also submitted a fix to the node js implementation so that get Report doesn't take long times if excludeNetwork is true, at which point we don't need to cache getReport early, just caching it after the first call will be enough

[IngFu]

Many projects have more than 100 dependencies. In my opinion it makes sense to limit the getReport call to one time as it performs other tasks as well and the system properties wont chance during installation.

[Tofandel]

@reggi Any chance you can review and merge this quickly as this issue is serious (can completely hog CI servers) and prevents npm install or npm ugrade from working and it's really affecting a lot of people
nodejs/docker-node#1946
npm/cli#7891
npm/cli#7900
npm/cli#7868
npm/cli#4028
npm/cli#7814

[wraithgar]

I don't think the approach of assigning our own attributes to process.report is wise. Ideally if we want to cache these values, we'd move that to a separate file that we can then mock in the tests to bypass the cache.

[Tofandel]

@wraithgar I added the ldd file check with tests and use a mocked file to bypass the cache and avoid assigning to process.report

[wraithgar]

We're VERY close imo. Just gotta get the coverage fixed and we can ship this. I'm holding off on the next npm release till this lands because I think it should be pretty soon.

[wraithgar]

The testing setup is a little hard to follow with the extra callbacks but I don't think it's a blocker. We can refactor that later.

pinging @H4ad in case they had any input before we land this (hopefully today)

[H4ad]

I can't approve but LGTM!

[wraithgar]

Huge thanks to @H4ad @styfle and @Tofandel. There was a lot of coordination done to tie all the similar issues together, and of course to get this fix out the door.