From b5522866ee3f54a8b1a8408ac7269df7557637b2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bazyli=20Brz=C3=B3ska?= Date: Tue, 6 Aug 2019 14:30:15 +0200 Subject: [PATCH 1/3] workaround node >=12.5.0 issue This is a workaround for an issue that occurs in node 12.5.0 and newer, described in detail here: https://github.com/nodejs/node/issues/29001 The behavior of the code in question is the same, however the change is enough to prevent locking the process into a "cancerous" state of memory growth. This is one of the weirdest bugs I've ever debugged. Under certain circumstances, Node process (starting from version 12.5.0, previous versions not affected), would not exit (even if we call `process.exit()` at the end of the file), and instead start to consume memory until it fills the whole available memory, using 100% CPU while at it. Here's a reproduction based on the `HasteMap`: https://gist.github.com/niieani/a527e9b2e6caf28e3021a50b938e4a91 Standard memory or CPU profiling doesn't show anything (literally, there are no function calls shown in the profiler). Dumping the heap snapshot results in a 36MB file, even when the memory consumption of the process is 10 GB. The backstory is that in our case [`jest`](https://github.com/facebook/jest) would not end AFTER all the tests have completed, and instead would start consuming huge amounts of memory (on CI, multiple processes from separate PRs would deplete the 128GB of RAM on the machine) so I started going down the rabbit hole to try and pinpoint what is the cause of the issue, which lead me to this minimal reproduction scenario, but also to discover all the quirks that helped to avoid triggering the scenario. --- packages/jest-haste-map/src/index.ts | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/packages/jest-haste-map/src/index.ts b/packages/jest-haste-map/src/index.ts index 70965909d79d..f37a4a4f18df 100644 --- a/packages/jest-haste-map/src/index.ts +++ b/packages/jest-haste-map/src/index.ts @@ -687,16 +687,18 @@ class HasteMap extends EventEmitter { } } - try { - await Promise.all(promises); - this._cleanup(); - hasteMap.map = map; - hasteMap.mocks = mocks; - return hasteMap; - } catch (error) { - this._cleanup(); - throw error; - } + return Promise.all(promises).then( + () => { + this._cleanup(); + hasteMap.map = map; + hasteMap.mocks = mocks; + return hasteMap; + }, + error => { + this._cleanup(); + throw error; + }, + ); } private _cleanup() { From 4ffdcd8fc0919e2b5f70ce3f368ed6aa9b39a3ca Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bazyli=20Brz=C3=B3ska?= Date: Tue, 6 Aug 2019 17:42:48 +0200 Subject: [PATCH 2/3] `_buildHasteMap` no longer needs to be an async method --- packages/jest-haste-map/src/index.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/jest-haste-map/src/index.ts b/packages/jest-haste-map/src/index.ts index f37a4a4f18df..4a14bebc2eee 100644 --- a/packages/jest-haste-map/src/index.ts +++ b/packages/jest-haste-map/src/index.ts @@ -642,7 +642,7 @@ class HasteMap extends EventEmitter { .then(workerReply, workerError); } - private async _buildHasteMap(data: { + private _buildHasteMap(data: { removedFiles: FileData; changedFiles?: FileData; hasteMap: InternalHasteMap; From 2cad529e6a2cfdd830a611853fa3c437e270dcb8 Mon Sep 17 00:00:00 2001 From: scotthovestadt Date: Thu, 15 Aug 2019 22:43:01 -0700 Subject: [PATCH 3/3] Update CHANGELOG.md --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6283466826be..d77ce40779ba 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -34,6 +34,7 @@ - `[jest-snapshots]` Fix test retries that contain snapshots ([#8629](https://github.com/facebook/jest/pull/8629)) - `[jest-mock]` Fix incorrect assignments when restoring mocks in instances where they originally didn't exist ([#8631](https://github.com/facebook/jest/pull/8631)) - `[expect]` Fix stack overflow when matching objects with circular references ([#8687](https://github.com/facebook/jest/pull/8687)) +- `[jest-haste-map]` Workaround a node >=12.5.0 bug that causes the process not to exit after tests have completed and cancerous memory growth ([#8787](https://github.com/facebook/jest/pull/8787)) ### Chore & Maintenance