-
Notifications
You must be signed in to change notification settings - Fork 30k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[v20.x] backport vm-related memory fixes #49874
[v20.x] backport vm-related memory fixes #49874
Conversation
Review requested:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
The
notable-change
Please suggest a text for the release notes if you'd like to include a more detailed summary, then proceed to update the PR description with the text or a link to the notable change suggested text comment. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Honestly, to me this seems very much fine to land without breaking the ABI version.
It's a minor ABI breakage, however, in that the V8 patches mostly just changes the type casts and the class hierarchy of a few V8 classes, the type cast changes would only incur check failures in debug mode
… if they fail, right? I think that’s fine because it only happens when casting to a value that isn’t actually of the expected type.
while the breakages introduced by class hierarchy change is implementation-defined, they are both POD types with no instance members, but theoretically this could still break the ABI.
I don’t think even a class layout change would break the ABI here? Users are not expected to subclass these types, I think it’s fair to call that unsupported, and the rest is all pointer accesses through Local<...>
anyway.
If regardless of that the concern about ABI breakage is still too big, you could always remove the inheritance of Script
and UnboundScript
from Data
and add static methods to convert between Local<Script>
and Local<Data>
, using V8’s own internal conversion methods, right? Then the classes would look exactly the same to consumers.
That sounds like a better idea. Actually on that note, I wonder if we can just add some new SetInternalField overloads specifically for v8::Module and v8::UnboundScript, now that could involve some pretty dumb copy-pasting but if we aren't going to update V8 on v20.x much it's probably okay anyway, and that probably reduces the likelihood of breakages to zero |
245596d
to
cbafa9c
Compare
cbafa9c
to
4eeaf62
Compare
Agreed, in this form this is definitely not an ABI or API breakage concern 🙂 |
Previously we simply create a lot of the target objects and check if the process crash due to OOM. Due to how we use emphemeron GC to handle memory management, which is inefficient but necessary for correctness, the tests can produce false positives as the GC isn't efficient enough to catch up with a very fast heap growth. This patch uses a new checkIfCollectable() utility to terminate the test early once we detect that any of the target object can actually be garbage collected. This should lower the chance of false positives. As a drive-by this also allows us to use setImmediate() to grow the heap even faster to make the tests run faster. Backport-PR-URL: #49874 PR-URL: #49671 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Michaël Zasso <[email protected]>
Similar to the test-vm-source-text-module-leak fix, use a snapshot to force a thorough GC in order to prevent false positives. Backport-PR-URL: #49874 PR-URL: #49710 Refs: nodejs/reliability#669 Reviewed-By: Franziska Hinkelmann <[email protected]> Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Rich Trott <[email protected]>
Landed in e49a573...be211ef |
Notable changes: deps: * add v8::Object::SetInternalFieldForNodeCore() (Joyee Cheung) #49874 doc: * deprecate `fs.F_OK`, `fs.R_OK`, `fs.W_OK`, `fs.X_OK` (Livia Medeiros) #49683 * deprecate `util.toUSVString` (Yagiz Nizipli) #49725 * deprecate calling `promisify` on a function that returns a promise (Antoine du Hamel) #49647 esm: * set all hooks as release candidate (Geoffrey Booth) #49597 module: * fix the leak in SourceTextModule and ContextifySript (Joyee Cheung) #48510 * fix leak of vm.SyntheticModule (Joyee Cheung) #48510 * use symbol in WeakMap to manage host defined options (Joyee Cheung) #48510 src: * (SEMVER-MINOR) allow embedders to override NODE_MODULE_VERSION (Cheng Zhao) #49279 stream: * use bitmap in writable state (Raz Luvaton) #49834 * use bitmap in readable state (Benjamin Gruenbaum) #49745 * improve webstream readable async iterator performance (Raz Luvaton) #49662 test_runner: * (SEMVER-MINOR) accept `testOnly` in `run` (Moshe Atlow) #49753 * (SEMVER-MINOR) add junit reporter (Moshe Atlow) #49614 PR-URL: #49932
Notable changes: deps: * add v8::Object::SetInternalFieldForNodeCore() (Joyee Cheung) #49874 doc: * deprecate `fs.F_OK`, `fs.R_OK`, `fs.W_OK`, `fs.X_OK` (Livia Medeiros) #49683 * deprecate `util.toUSVString` (Yagiz Nizipli) #49725 * deprecate calling `promisify` on a function that returns a promise (Antoine du Hamel) #49647 esm: * set all hooks as release candidate (Geoffrey Booth) #49597 module: * fix the leak in SourceTextModule and ContextifySript (Joyee Cheung) #48510 * fix leak of vm.SyntheticModule (Joyee Cheung) #48510 * use symbol in WeakMap to manage host defined options (Joyee Cheung) #48510 src: * (SEMVER-MINOR) allow embedders to override NODE_MODULE_VERSION (Cheng Zhao) #49279 stream: * use bitmap in writable state (Raz Luvaton) #49834 * use bitmap in readable state (Benjamin Gruenbaum) #49745 * improve webstream readable async iterator performance (Raz Luvaton) #49662 test_runner: * (SEMVER-MINOR) accept `testOnly` in `run` (Moshe Atlow) #49753 * (SEMVER-MINOR) add junit reporter (Moshe Atlow) #49614 PR-URL: #49932
Notable changes: deps: * add v8::Object::SetInternalFieldForNodeCore() (Joyee Cheung) #49874 doc: * deprecate `fs.F_OK`, `fs.R_OK`, `fs.W_OK`, `fs.X_OK` (Livia Medeiros) #49683 * deprecate `util.toUSVString` (Yagiz Nizipli) #49725 * deprecate calling `promisify` on a function that returns a promise (Antoine du Hamel) #49647 esm: * set all hooks as release candidate (Geoffrey Booth) #49597 module: * fix the leak in SourceTextModule and ContextifySript (Joyee Cheung) #48510 * fix leak of vm.SyntheticModule (Joyee Cheung) #48510 * use symbol in WeakMap to manage host defined options (Joyee Cheung) #48510 src: * (SEMVER-MINOR) allow embedders to override NODE_MODULE_VERSION (Cheng Zhao) #49279 stream: * use bitmap in writable state (Raz Luvaton) #49834 * use bitmap in readable state (Benjamin Gruenbaum) #49745 * improve webstream readable async iterator performance (Raz Luvaton) #49662 test_runner: * (SEMVER-MINOR) accept `testOnly` in `run` (Moshe Atlow) #49753 * (SEMVER-MINOR) add junit reporter (Moshe Atlow) #49614 PR-URL: #49932
Notable changes: deps: * add v8::Object::SetInternalFieldForNodeCore() (Joyee Cheung) nodejs#49874 doc: * deprecate `fs.F_OK`, `fs.R_OK`, `fs.W_OK`, `fs.X_OK` (Livia Medeiros) nodejs#49683 * deprecate `util.toUSVString` (Yagiz Nizipli) nodejs#49725 * deprecate calling `promisify` on a function that returns a promise (Antoine du Hamel) nodejs#49647 esm: * set all hooks as release candidate (Geoffrey Booth) nodejs#49597 module: * fix the leak in SourceTextModule and ContextifySript (Joyee Cheung) nodejs#48510 * fix leak of vm.SyntheticModule (Joyee Cheung) nodejs#48510 * use symbol in WeakMap to manage host defined options (Joyee Cheung) nodejs#48510 src: * (SEMVER-MINOR) allow embedders to override NODE_MODULE_VERSION (Cheng Zhao) nodejs#49279 stream: * use bitmap in writable state (Raz Luvaton) nodejs#49834 * use bitmap in readable state (Benjamin Gruenbaum) nodejs#49745 * improve webstream readable async iterator performance (Raz Luvaton) nodejs#49662 test_runner: * (SEMVER-MINOR) accept `testOnly` in `run` (Moshe Atlow) nodejs#49753 * (SEMVER-MINOR) add junit reporter (Moshe Atlow) nodejs#49614 PR-URL: nodejs#49932
This is a non-ABI breaking solution for v8/v8@b60a03d and v8/v8@0aa622e which are necessary for backporting vm-related memory fixes to v18.x. PR-URL: nodejs#49874 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>
* chore: upgrade to Node.js v20 * src: allow embedders to override NODE_MODULE_VERSION nodejs/node#49279 * src: fix missing trailing , nodejs/node#46909 * src,tools: initialize cppgc nodejs/node#45704 * tools: allow passing absolute path of config.gypi in js2c nodejs/node#49162 * tools: port js2c.py to C++ nodejs/node#46997 * doc,lib: disambiguate the old term, NativeModule nodejs/node#45673 * chore: fixup Node.js BSSL tests * nodejs/node#49492 * nodejs/node#44498 * deps: upgrade to libuv 1.45.0 nodejs/node#48078 * deps: update V8 to 10.7 nodejs/node#44741 * test: use gcUntil() in test-v8-serialize-leak nodejs/node#49168 * module: make CJS load from ESM loader nodejs/node#47999 * src: make BuiltinLoader threadsafe and non-global nodejs/node#45942 * chore: address changes to CJS/ESM loading * module: make CJS load from ESM loader (nodejs/node#47999) * lib: improve esm resolve performance (nodejs/node#46652) * bootstrap: optimize modules loaded in the built-in snapshot nodejs/node#45849 * test: mark test-runner-output as flaky nodejs/node#49854 * lib: lazy-load deps in modules/run_main.js nodejs/node#45849 * url: use private properties for brand check nodejs/node#46904 * test: refactor `test-node-output-errors` nodejs/node#48992 * assert: deprecate callTracker nodejs/node#47740 * src: cast v8::Object::GetInternalField() return value to v8::Value nodejs/node#48943 * test: adapt test-v8-stats for V8 update nodejs/node#45230 * tls: ensure TLS Sockets are closed if the underlying wrap closes nodejs/node#49327 * test: deflake test-tls-socket-close nodejs/node#49575 * net: fix crash due to simultaneous close/shutdown on JS Stream Sockets nodejs/node#49400 * net: use asserts in JS Socket Stream to catch races in future nodejs/node#49400 * lib: fix BroadcastChannel initialization location nodejs/node#46864 * src: create BaseObject with node::Realm nodejs/node#44348 * src: implement DataQueue and non-memory resident Blob nodejs/node#45258 * sea: add support for V8 bytecode-only caching nodejs/node#48191 * chore: fixup patch indices * gyp: put filenames in variables nodejs/node#46965 * build: modify js2c.py into GN executable * fix: (WIP) handle string replacement of fs -> original-fs * [v20.x] backport vm-related memory fixes nodejs/node#49874 * src: make BuiltinLoader threadsafe and non-global nodejs/node#45942 * src: avoid copying string in fs_permission nodejs/node#47746 * look upon my works ye mighty and dispair * chore: patch cleanup * [api] Remove AllCan Read/Write https://chromium-review.googlesource.com/c/v8/v8/+/5006387 * fix: missing include for NODE_EXTERN * chore: fixup patch indices * fix: fail properly when js2c fails in Node.js * build: fix js2c root_gen_dir * fix: lib/fs.js -> lib/original-fs.js * build: fix original-fs file xforms * fixup! module: make CJS load from ESM loader * build: get rid of CppHeap for now * build: add patch to prevent extra fs lookup on esm load * build: greatly simplify js2c modifications Moves our original-fs modifications back into a super simple python script action, wires up the output of that action into our call to js2c * chore: update to handle moved internal/modules/helpers file * test: update @types/node test * feat: enable preventing cppgc heap creation * feat: optionally prevent calling V8::EnableWebAssemblyTrapHandler * fix: no cppgc initialization in the renderer * gyp: put filenames in variables nodejs/node#46965 * test: disable single executable tests * fix: nan tests failing on node headers missing file * tls,http2: send fatal alert on ALPN mismatch nodejs/node#44031 * test: disable snapshot tests * nodejs/node#47887 * nodejs/node#49684 * nodejs/node#44193 * build: use deps/v8 for v8/tools Node.js hard depends on these in their builtins * test: fix edge snapshot stack traces nodejs/node#49659 * build: remove js2c //base dep * build: use electron_js2c_toolchain to build node_js2c * fix: don't create SafeSet outside packageResolve Fixes failure in parallel/test-require-delete-array-iterator: === release test-require-delete-array-iterator === Path: parallel/test-require-delete-array-iterator node:internal/per_context/primordials:426 constructor(i) { super(i); } // eslint-disable-line no-useless-constructor ^ TypeError: object is not iterable (cannot read property Symbol(Symbol.iterator)) at new Set (<anonymous>) at new SafeSet (node:internal/per_context/primordials:426:22) * fix: failing crashReporter tests on Linux These were failing because our change from node::InitializeNodeWithArgs to node::InitializeOncePerProcess meant that we now inadvertently called PlatformInit, which reset signal handling. This meant that our intentional crash function ElectronBindings::Crash no longer worked and the renderer process no longer crashed when process.crash() was called. We don't want to use Node.js' default signal handling in the renderer process, so we disable it by passing kNoDefaultSignalHandling to node::InitializeOncePerProcess. * build: only create cppgc heap on non-32 bit platforms * chore: clean up util:CompileAndCall * src: fix compatility with upcoming V8 12.1 APIs nodejs/node#50709 * fix: use thread_local BuiltinLoader * chore: fixup v8 patch indices --------- Co-authored-by: Keeley Hammond <[email protected]> Co-authored-by: Samuel Attard <[email protected]>
This is a non-ABI breaking solution for v8/v8@b60a03d and v8/v8@0aa622e which are necessary for backporting vm-related memory fixes to v18.x. PR-URL: nodejs#49874 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>
* chore: upgrade to Node.js v20 * src: allow embedders to override NODE_MODULE_VERSION nodejs/node#49279 * src: fix missing trailing , nodejs/node#46909 * src,tools: initialize cppgc nodejs/node#45704 * tools: allow passing absolute path of config.gypi in js2c nodejs/node#49162 * tools: port js2c.py to C++ nodejs/node#46997 * doc,lib: disambiguate the old term, NativeModule nodejs/node#45673 * chore: fixup Node.js BSSL tests * nodejs/node#49492 * nodejs/node#44498 * deps: upgrade to libuv 1.45.0 nodejs/node#48078 * deps: update V8 to 10.7 nodejs/node#44741 * test: use gcUntil() in test-v8-serialize-leak nodejs/node#49168 * module: make CJS load from ESM loader nodejs/node#47999 * src: make BuiltinLoader threadsafe and non-global nodejs/node#45942 * chore: address changes to CJS/ESM loading * module: make CJS load from ESM loader (nodejs/node#47999) * lib: improve esm resolve performance (nodejs/node#46652) * bootstrap: optimize modules loaded in the built-in snapshot nodejs/node#45849 * test: mark test-runner-output as flaky nodejs/node#49854 * lib: lazy-load deps in modules/run_main.js nodejs/node#45849 * url: use private properties for brand check nodejs/node#46904 * test: refactor `test-node-output-errors` nodejs/node#48992 * assert: deprecate callTracker nodejs/node#47740 * src: cast v8::Object::GetInternalField() return value to v8::Value nodejs/node#48943 * test: adapt test-v8-stats for V8 update nodejs/node#45230 * tls: ensure TLS Sockets are closed if the underlying wrap closes nodejs/node#49327 * test: deflake test-tls-socket-close nodejs/node#49575 * net: fix crash due to simultaneous close/shutdown on JS Stream Sockets nodejs/node#49400 * net: use asserts in JS Socket Stream to catch races in future nodejs/node#49400 * lib: fix BroadcastChannel initialization location nodejs/node#46864 * src: create BaseObject with node::Realm nodejs/node#44348 * src: implement DataQueue and non-memory resident Blob nodejs/node#45258 * sea: add support for V8 bytecode-only caching nodejs/node#48191 * chore: fixup patch indices * gyp: put filenames in variables nodejs/node#46965 * build: modify js2c.py into GN executable * fix: (WIP) handle string replacement of fs -> original-fs * [v20.x] backport vm-related memory fixes nodejs/node#49874 * src: make BuiltinLoader threadsafe and non-global nodejs/node#45942 * src: avoid copying string in fs_permission nodejs/node#47746 * look upon my works ye mighty and dispair * chore: patch cleanup * [api] Remove AllCan Read/Write https://chromium-review.googlesource.com/c/v8/v8/+/5006387 * fix: missing include for NODE_EXTERN * chore: fixup patch indices * fix: fail properly when js2c fails in Node.js * build: fix js2c root_gen_dir * fix: lib/fs.js -> lib/original-fs.js * build: fix original-fs file xforms * fixup! module: make CJS load from ESM loader * build: get rid of CppHeap for now * build: add patch to prevent extra fs lookup on esm load * build: greatly simplify js2c modifications Moves our original-fs modifications back into a super simple python script action, wires up the output of that action into our call to js2c * chore: update to handle moved internal/modules/helpers file * test: update @types/node test * feat: enable preventing cppgc heap creation * feat: optionally prevent calling V8::EnableWebAssemblyTrapHandler * fix: no cppgc initialization in the renderer * gyp: put filenames in variables nodejs/node#46965 * test: disable single executable tests * fix: nan tests failing on node headers missing file * tls,http2: send fatal alert on ALPN mismatch nodejs/node#44031 * test: disable snapshot tests * nodejs/node#47887 * nodejs/node#49684 * nodejs/node#44193 * build: use deps/v8 for v8/tools Node.js hard depends on these in their builtins * test: fix edge snapshot stack traces nodejs/node#49659 * build: remove js2c //base dep * build: use electron_js2c_toolchain to build node_js2c * fix: don't create SafeSet outside packageResolve Fixes failure in parallel/test-require-delete-array-iterator: === release test-require-delete-array-iterator === Path: parallel/test-require-delete-array-iterator node:internal/per_context/primordials:426 constructor(i) { super(i); } // eslint-disable-line no-useless-constructor ^ TypeError: object is not iterable (cannot read property Symbol(Symbol.iterator)) at new Set (<anonymous>) at new SafeSet (node:internal/per_context/primordials:426:22) * fix: failing crashReporter tests on Linux These were failing because our change from node::InitializeNodeWithArgs to node::InitializeOncePerProcess meant that we now inadvertently called PlatformInit, which reset signal handling. This meant that our intentional crash function ElectronBindings::Crash no longer worked and the renderer process no longer crashed when process.crash() was called. We don't want to use Node.js' default signal handling in the renderer process, so we disable it by passing kNoDefaultSignalHandling to node::InitializeOncePerProcess. * build: only create cppgc heap on non-32 bit platforms * chore: clean up util:CompileAndCall * src: fix compatility with upcoming V8 12.1 APIs nodejs/node#50709 * fix: use thread_local BuiltinLoader * chore: fixup v8 patch indices --------- Co-authored-by: Keeley Hammond <[email protected]> Co-authored-by: Samuel Attard <[email protected]>
This is a non-ABI breaking solution for v8/v8@b60a03d and v8/v8@0aa622e which are necessary for backporting vm-related memory fixes to v18.x. PR-URL: #49874 Backport-PR-URL: #51004 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>
This PR contains the following updates: | Package | Update | Change | |---|---|---| | [node](https://nodejs.org) ([source](https://github.com/nodejs/node)) | minor | `18.19.1` -> `18.20.0` | --- ### Release Notes <details> <summary>nodejs/node (node)</summary> ### [`v18.20.0`](https://github.com/nodejs/node/releases/tag/v18.20.0): 2024-03-26, Version 18.20.0 'Hydrogen' (LTS), @​richardlau [Compare Source](nodejs/node@v18.19.1...v18.20.0) ##### Notable Changes ##### Added support for import attributes Support has been added for import attributes, to replace the old import assertions syntax. This will aid migration by making the new syntax available across all currently supported Node.js release lines. This adds the `with` keyword which should be used in place of the previous `assert` keyword, which will be removed in a future semver-major Node.js release. For example, ```console import "foo" assert { ... } ``` should be replaced with ```console import "foo" with { ... } ``` For more details, see - [#​50134](nodejs/node#50134) - [#​51622](nodejs/node#51622) Contributed by Nicolò Ribaudo in [#​51136](nodejs/node#51136) and Antoine du Hamel in [#​50140](nodejs/node#50140). ##### Doc deprecation for `dirent.path` Please use newly added `dirent.parentPath` instead. Contributed by Antoine du Hamel in [#​50976](nodejs/node#50976) and [#​51020](nodejs/node#51020). ##### Experimental node-api feature flags Introduces an experimental feature to segregate finalizers that affect GC state. A new type called `node_api_nogc_env` has been introduced as the const version of `napi_env` and `node_api_nogc_finalize` as a variant of `napi_finalize` that accepts a `node_api_nogc_env` as its first argument. This feature can be turned off by defining `NODE_API_EXPERIMENTAL_NOGC_ENV_OPT_OUT`. Contributed by Gabriel Schulhof in [#​50060](nodejs/node#50060). ##### Root certificates updated to NSS 3.98 Certificates added: - Telekom Security TLS ECC Root 2020 - Telekom Security TLS RSA Root 2023 Certificates removed: - Security Communication Root CA ##### Updated dependencies - ada updated to 2.7.6. - base64 updated to 0.5.2. - c-ares updated to 1.27.0. - corepack updated to 0.25.2. - ICU updated to 74.2. Includes CLDR 44.1 and Unicode 15.1. - npm updated to 10.5.0. Fixes a regression in signals not being passed onto child processes. - simdutf8 updated to 4.0.8. - Timezone updated to 2024a. - zlib updated to 1.3.0.1-motley-40e35a7. ##### vm: fix V8 compilation cache support for vm.Script Previously repeated compilation of the same source code using `vm.Script` stopped hitting the V8 compilation cache after v16.x when support for `importModuleDynamically` was added to `vm.Script`, resulting in a performance regression that blocked users (in particular Jest users) from upgrading from v16.x. The recent fixes allow the compilation cache to be hit again for `vm.Script` when `--experimental-vm-modules` is not used even in the presence of the `importModuleDynamically` option, so that users affected by the performance regression can now upgrade. Ongoing work is also being done to enable compilation cache support for `vm.CompileFunction`. Contributed by Joyee Cheung in [#​49950](nodejs/node#49950) and [#​50137](nodejs/node#50137). ##### Commits - \[[`c70383b8d4`](nodejs/node@c70383b8d4)] - **build**: support Python 3.12 (Shi Pujin) [#​50209](nodejs/node#50209) - \[[`4b960c3a4a`](nodejs/node@4b960c3a4a)] - **build**: fix incorrect g++ warning message (Richard Lau) [#​51695](nodejs/node#51695) - \[[`8fdea67694`](nodejs/node@8fdea67694)] - **crypto**: update root certificates to NSS 3.98 (Node.js GitHub Bot) [#​51794](nodejs/node#51794) - \[[`812b126dd9`](nodejs/node@812b126dd9)] - **deps**: V8: cherry-pick [`d90d453`](nodejs/node@d90d4533b053) (Michaël Zasso) [#​50077](nodejs/node#50077) - \[[`9ab8c3db87`](nodejs/node@9ab8c3db87)] - **deps**: update c-ares to 1.27.0 (Node.js GitHub Bot) [#​51846](nodejs/node#51846) - \[[`c688680387`](nodejs/node@c688680387)] - **deps**: update c-ares to 1.26.0 (Node.js GitHub Bot) [#​51582](nodejs/node#51582) - \[[`9498ac8a47`](nodejs/node@9498ac8a47)] - **deps**: compile c-ares with C11 support (Michaël Zasso) [#​51410](nodejs/node#51410) - \[[`8fb743642f`](nodejs/node@8fb743642f)] - **deps**: update c-ares to 1.25.0 (Node.js GitHub Bot) [#​51385](nodejs/node#51385) - \[[`7bea2d7c12`](nodejs/node@7bea2d7c12)] - **deps**: update zlib to 1.3.0.1-motley-40e35a7 (Node.js GitHub Bot) [#​51274](nodejs/node#51274) - \[[`57a38c8f75`](nodejs/node@57a38c8f75)] - **deps**: update zlib to 1.3.0.1-motley-dd5fc13 (Node.js GitHub Bot) [#​51105](nodejs/node#51105) - \[[`b0ca084a6b`](nodejs/node@b0ca084a6b)] - **deps**: update zlib to 1.3-22124f5 (Node.js GitHub Bot) [#​50910](nodejs/node#50910) - \[[`4b43823f37`](nodejs/node@4b43823f37)] - **deps**: update zlib to 1.2.13.1-motley-5daffc7 (Node.js GitHub Bot) [#​50803](nodejs/node#50803) - \[[`f0da591812`](nodejs/node@f0da591812)] - **deps**: update zlib to 1.2.13.1-motley-dfc48fc (Node.js GitHub Bot) [#​50456](nodejs/node#50456) - \[[`16d28a883a`](nodejs/node@16d28a883a)] - **deps**: update base64 to 0.5.2 (Node.js GitHub Bot) [#​51455](nodejs/node#51455) - \[[`13a9e81cb6`](nodejs/node@13a9e81cb6)] - **deps**: update base64 to 0.5.1 (Node.js GitHub Bot) [#​50629](nodejs/node#50629) - \[[`b4502d3ac5`](nodejs/node@b4502d3ac5)] - **deps**: update simdutf to 4.0.8 (Node.js GitHub Bot) [#​51000](nodejs/node#51000) - \[[`183cf8a74a`](nodejs/node@183cf8a74a)] - **deps**: update simdutf to 4.0.4 (Node.js GitHub Bot) [#​50772](nodejs/node#50772) - \[[`11ba8593ea`](nodejs/node@11ba8593ea)] - **deps**: update ada to 2.7.6 (Node.js GitHub Bot) [#​51542](nodejs/node#51542) - \[[`73a946d55c`](nodejs/node@73a946d55c)] - **deps**: update ada to 2.7.5 (Node.js GitHub Bot) [#​51542](nodejs/node#51542) - \[[`cc434c1a39`](nodejs/node@cc434c1a39)] - **deps**: update ada to 2.7.4 (Node.js GitHub Bot) [#​50815](nodejs/node#50815) - \[[`3a3808a6ae`](nodejs/node@3a3808a6ae)] - **deps**: upgrade npm to 10.5.0 (npm team) [#​51913](nodejs/node#51913) - \[[`c8876d765c`](nodejs/node@c8876d765c)] - **deps**: upgrade npm to 10.3.0 (npm team) [#​51431](nodejs/node#51431) - \[[`5aec3af460`](nodejs/node@5aec3af460)] - **deps**: update corepack to 0.25.2 (Node.js GitHub Bot) [#​51810](nodejs/node#51810) - \[[`a593985326`](nodejs/node@a593985326)] - **deps**: update corepack to 0.24.1 (Node.js GitHub Bot) [#​51459](nodejs/node#51459) - \[[`d1a9237bf5`](nodejs/node@d1a9237bf5)] - **deps**: update corepack to 0.24.0 (Node.js GitHub Bot) [#​51318](nodejs/node#51318) - \[[`adac0c7a63`](nodejs/node@adac0c7a63)] - **deps**: update corepack to 0.23.0 (Node.js GitHub Bot) [#​50563](nodejs/node#50563) - \[[`4a6f83e32a`](nodejs/node@4a6f83e32a)] - **deps**: escape Python strings correctly (Michaël Zasso) [#​50695](nodejs/node#50695) - \[[`c13969e52a`](nodejs/node@c13969e52a)] - **deps**: V8: cherry-pick [`ea996ad`](nodejs/node@ea996ad04a68) (Nicolò Ribaudo) [#​51136](nodejs/node#51136) - \[[`6fbf0ba5c3`](nodejs/node@6fbf0ba5c3)] - **deps**: V8: cherry-pick [`a0fd320`](nodejs/node@a0fd3209dda8) (Nicolò Ribaudo) [#​51136](nodejs/node#51136) - \[[`68fd7516e1`](nodejs/node@68fd7516e1)] - **deps**: update timezone to 2024a (Michaël Zasso) [#​51723](nodejs/node#51723) - \[[`f9b229ebe1`](nodejs/node@f9b229ebe1)] - **deps**: update icu to 74.2 (Michaël Zasso) [#​51723](nodejs/node#51723) - \[[`90c73d2eb4`](nodejs/node@90c73d2eb4)] - **deps**: update timezone to 2023d (Node.js GitHub Bot) [#​51461](nodejs/node#51461) - \[[`2a2bf57028`](nodejs/node@2a2bf57028)] - **deps**: update icu to 74.1 (Node.js GitHub Bot) [#​50515](nodejs/node#50515) - \[[`425e011e52`](nodejs/node@425e011e52)] - **deps**: add v8::Object::SetInternalFieldForNodeCore() (Joyee Cheung) [#​49874](nodejs/node#49874) - \[[`58c70344a2`](nodejs/node@58c70344a2)] - **deps**: V8: cherry-pick [`705e374`](nodejs/node@705e374124ae) (Joyee Cheung) [#​51004](nodejs/node#51004) - \[[`b0e88899e1`](nodejs/node@b0e88899e1)] - **deps**: V8: cherry-pick [`1fada6b`](nodejs/node@1fada6b36f8d) (Joyee Cheung) [#​51004](nodejs/node#51004) - \[[`d87a810b81`](nodejs/node@d87a810b81)] - **deps**: V8: cherry-pick [`3dd9576`](nodejs/node@3dd9576ce336) (Joyee Cheung) [#​51004](nodejs/node#51004) - \[[`6d50966876`](nodejs/node@6d50966876)] - **deps**: V8: cherry-pick [`94e8282`](nodejs/node@94e8282325a1) (Joyee Cheung) [#​51004](nodejs/node#51004) - \[[`fafbacdfec`](nodejs/node@fafbacdfec)] - **deps**: V8: cherry-pick [`9a98f96`](nodejs/node@9a98f96b6d68) (Joyee Cheung) [#​51004](nodejs/node#51004) - \[[`d4a530ed8d`](nodejs/node@d4a530ed8d)] - **deps**: V8: cherry-pick [`7f5daed`](nodejs/node@7f5daed62d47) (Joyee Cheung) [#​51004](nodejs/node#51004) - \[[`1ce901b164`](nodejs/node@1ce901b164)] - **deps**: V8: cherry-pick [`c400af4`](nodejs/node@c400af48b5ef) (Joyee Cheung) [#​51004](nodejs/node#51004) - \[[`f232064f35`](nodejs/node@f232064f35)] - **doc**: fix historical experimental fetch flag (Kenrick) [#​51506](nodejs/node#51506) - \[[`194ff6a40f`](nodejs/node@194ff6a40f)] - **(SEMVER-MINOR)** **doc**: add deprecation notice to `dirent.path` (Antoine du Hamel) [#​50976](nodejs/node#50976) - \[[`0f09267dc6`](nodejs/node@0f09267dc6)] - **(SEMVER-MINOR)** **doc**: deprecate `dirent.path` (Antoine du Hamel) [#​50976](nodejs/node#50976) - \[[`8bfb8f5b2f`](nodejs/node@8bfb8f5b2f)] - **doc,crypto**: further clarify RSA_PKCS1\_PADDING support (Tobias Nießen) [#​51799](nodejs/node#51799) - \[[`c7baf7b274`](nodejs/node@c7baf7b274)] - **doc,crypto**: add changelog and note about disabled RSA_PKCS1\_PADDING (Filip Skokan) [#​51782](nodejs/node#51782) - \[[`a193be3dc2`](nodejs/node@a193be3dc2)] - **esm**: use import attributes instead of import assertions (Antoine du Hamel) [#​50140](nodejs/node#50140) - \[[`26e8f7793e`](nodejs/node@26e8f7793e)] - **(SEMVER-MINOR)** **fs**: introduce `dirent.parentPath` (Antoine du Hamel) [#​50976](nodejs/node#50976) - \[[`5b5e5192f7`](nodejs/node@5b5e5192f7)] - **lib**: fix compileFunction throws range error for negative numbers (Jithil P Ponnan) [#​49855](nodejs/node#49855) - \[[`7552de6806`](nodejs/node@7552de6806)] - **module**: fix the leak in SourceTextModule and ContextifySript (Joyee Cheung) [#​48510](nodejs/node#48510) - \[[`2e05cf1c60`](nodejs/node@2e05cf1c60)] - **module**: fix leak of vm.SyntheticModule (Joyee Cheung) [#​48510](nodejs/node#48510) - \[[`a86a2e14a3`](nodejs/node@a86a2e14a3)] - **module**: use symbol in WeakMap to manage host defined options (Joyee Cheung) [#​48510](nodejs/node#48510) - \[[`32906ddcac`](nodejs/node@32906ddcac)] - **node-api**: segregate nogc APIs from rest via type system (Gabriel Schulhof) [#​50060](nodejs/node#50060) - \[[`1aa71c26ff`](nodejs/node@1aa71c26ff)] - **node-api**: factor out common code into macros (Gabriel Schulhof) [#​50664](nodejs/node#50664) - \[[`3d0b233f52`](nodejs/node@3d0b233f52)] - **node-api**: introduce experimental feature flags (Gabriel Schulhof) [#​50991](nodejs/node#50991) - \[[`96514a8b9f`](nodejs/node@96514a8b9f)] - **src**: iterate on import attributes array correctly (Michaël Zasso) [#​50703](nodejs/node#50703) - \[[`2c2892bf88`](nodejs/node@2c2892bf88)] - **src**: set ModuleWrap internal fields only once (Joyee Cheung) [#​49391](nodejs/node#49391) - \[[`ff334cb774`](nodejs/node@ff334cb774)] - **src**: cast v8::Object::GetInternalField() return value to v8::Value (Joyee Cheung) [#​48943](nodejs/node#48943) - \[[`270b519971`](nodejs/node@270b519971)] - **stream**: do not defer construction by one microtick (Matteo Collina) [#​52005](nodejs/node#52005) - \[[`95d7a75084`](nodejs/node@95d7a75084)] - **test**: fix dns test case failures after c-ares update to 1.21.0+ (Brad House) [#​50743](nodejs/node#50743) - \[[`cd613e5167`](nodejs/node@cd613e5167)] - **test**: handle relative https redirect (Richard Lau) [#​51121](nodejs/node#51121) - \[[`40f10eafcf`](nodejs/node@40f10eafcf)] - **test**: fix `internet/test-inspector-help-page` (Richard Lau) [#​51693](nodejs/node#51693) - \[[`5e426511b1`](nodejs/node@5e426511b1)] - **test**: deflake test-vm-contextified-script-leak (Joyee Cheung) [#​49710](nodejs/node#49710) - \[[`0b156c6d28`](nodejs/node@0b156c6d28)] - **test**: use checkIfCollectable in vm leak tests (Joyee Cheung) [#​49671](nodejs/node#49671) - \[[`1586c11b3c`](nodejs/node@1586c11b3c)] - **test**: add checkIfCollectable to test/common/gc.js (Joyee Cheung) [#​49671](nodejs/node#49671) - \[[`902d8b3d4b`](nodejs/node@902d8b3d4b)] - **test**: fix flaky http-chunk-extensions-limit test (Ethan Arrowood) [#​51943](nodejs/node#51943) - \[[`1743d2bdc1`](nodejs/node@1743d2bdc1)] - **test**: test surrogate pair filenames on windows (Mert Can Altın) [#​51800](nodejs/node#51800) - \[[`1c1a7ec22d`](nodejs/node@1c1a7ec22d)] - **test**: increase platform timeout zlib-brotli-16gb (Rafael Gonzaga) [#​51792](nodejs/node#51792) - \[[`931d02fe3e`](nodejs/node@931d02fe3e)] - **test, v8**: fix wrong import attributes test (Nicolò Ribaudo) [#​52184](nodejs/node#52184) - \[[`d9ea6c1f8d`](nodejs/node@d9ea6c1f8d)] - **tls**: fix order of setting cipher before setting cert and key (Kumar Rishav) [#​50186](nodejs/node#50186) - \[[`3184befa2e`](nodejs/node@3184befa2e)] - **tools**: fix update-icu.sh (Michaël Zasso) [#​51723](nodejs/node#51723) - \[[`06646e11be`](nodejs/node@06646e11be)] - **(SEMVER-MINOR)** **vm**: use import attributes instead of import assertions (Antoine du Hamel) [#​50141](nodejs/node#50141) - \[[`fe66e9d06e`](nodejs/node@fe66e9d06e)] - **vm**: reject in importModuleDynamically without --experimental-vm-modules (Joyee Cheung) [#​50137](nodejs/node#50137) - \[[`052e095c6b`](nodejs/node@052e095c6b)] - **vm**: use internal versions of compileFunction and Script (Joyee Cheung) [#​50137](nodejs/node#50137) - \[[`9f7899ed0a`](nodejs/node@9f7899ed0a)] - **vm**: unify host-defined option generation in vm.compileFunction (Joyee Cheung) [#​50137](nodejs/node#50137) - \[[`6291c107d0`](nodejs/node@6291c107d0)] - **vm**: use default HDO when importModuleDynamically is not set (Joyee Cheung) [#​49950](nodejs/node#49950) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIwLjAuMC1zZW1hbnRpYy1yZWxlYXNlIiwidXBkYXRlZEluVmVyIjoiMC4wLjAtc2VtYW50aWMtcmVsZWFzZSIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==--> Reviewed-on: https://harton.dev/cinder/cinder-space/pulls/25 Co-authored-by: Renovate Bot <[email protected]> Co-committed-by: Renovate Bot <[email protected]>
Notable changes: deps: * add v8::Object::SetInternalFieldForNodeCore() (Joyee Cheung) nodejs#49874 doc: * deprecate `fs.F_OK`, `fs.R_OK`, `fs.W_OK`, `fs.X_OK` (Livia Medeiros) nodejs#49683 * deprecate `util.toUSVString` (Yagiz Nizipli) nodejs#49725 * deprecate calling `promisify` on a function that returns a promise (Antoine du Hamel) nodejs#49647 esm: * set all hooks as release candidate (Geoffrey Booth) nodejs#49597 module: * fix the leak in SourceTextModule and ContextifySript (Joyee Cheung) nodejs#48510 * fix leak of vm.SyntheticModule (Joyee Cheung) nodejs#48510 * use symbol in WeakMap to manage host defined options (Joyee Cheung) nodejs#48510 src: * (SEMVER-MINOR) allow embedders to override NODE_MODULE_VERSION (Cheng Zhao) nodejs#49279 stream: * use bitmap in writable state (Raz Luvaton) nodejs#49834 * use bitmap in readable state (Benjamin Gruenbaum) nodejs#49745 * improve webstream readable async iterator performance (Raz Luvaton) nodejs#49662 test_runner: * (SEMVER-MINOR) accept `testOnly` in `run` (Moshe Atlow) nodejs#49753 * (SEMVER-MINOR) add junit reporter (Moshe Atlow) nodejs#49614 PR-URL: nodejs#49932
This is a non-ABI breaking solution for v8/v8@b60a03d and v8/v8@0aa622e which are necessary for backporting vm-related memory fixes to v18.x. PR-URL: nodejs#49874 Backport-PR-URL: nodejs#51004 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>
This includes multiple PRs for the fixes and the tests related to vm memory issues.
To avoid breaking the ABI on v20.x, instead of backporting #49491 and #49419, we just introduce an completely new API
v8::Object::SetInternalFieldForNodeCore()
to our v20.x fork of V8 which can be used to add internal references tov8::UnboundScript
andv8::Module
, and use that instead to backport #48510 - this should be fine if we add enough warning to tell embedders and addon authors not to use them, and we will not update the V8 fork in v20.x often so it is unlikely to create conflicts.deps: add v8::Object::SetInternalFieldForNodeCore()
This is a non-ABI breaking solution for
v8/v8@b60a03d
and
v8/v8@0aa622e
which are necessary for backporting vm-related memory fixes to v20.x.
module: use symbol in WeakMap to manage host defined options
Previously when managing the importModuleDynamically callback of
vm.compileFunction(), we use an ID number as the host defined option
and maintain a per-Environment ID -> CompiledFnEntry map to retain
the top-level referrer function returned by vm.compileFunction() in
order to pass it back to the callback, but it would leak because with
how we used v8::Persistent to maintain this reference, V8 would not
be able to understand the cycle and would just think that the
CompiledFnEntry was supposed to live forever. We made an attempt
to make that reference known to V8 by making the CompiledFnEntry weak
and using a private symbol to make CompiledFnEntry strongly
references the top-level referrer function in
#46785, but that turned out to be
unsound, because the there's no guarantee that the top-level function
must be alive while import() can still be initiated from that
function, since V8 could discard the top-level function and only keep
inner functions alive, so relying on the top-level function to keep
the CompiledFnEntry alive could result in use-after-free which caused
a revert of that fix.
With this patch we use a symbol in the host defined options instead of
a number, because with the stage-3 symbol-as-weakmap-keys proposal
we could directly use that symbol to keep the referrer alive using a
WeakMap. As a bonus this also keeps the other kinds of referrers
alive as long as import() can still be initiated from that
Script/Module, so this also fixes the long-standing crash caused by
vm.Script being GC'ed too early when its importModuleDynamically
callback still needs it.
PR-URL: #48510
Refs: #44211
Refs: #42080
Refs: #47096
Refs: #43205
Refs: #38695
Reviewed-By: Antoine du Hamel [email protected]
Reviewed-By: Benjamin Gruenbaum [email protected]
Reviewed-By: Stephen Belanger [email protected]
module: fix leak of vm.SyntheticModule
Previously we maintain a strong persistent reference to the
ModuleWrap to retrieve the ID-to-ModuleWrap mapping from
the HostImportModuleDynamicallyCallback using the number ID
stored in the host-defined options. As a result the ModuleWrap
would be kept alive until the Environment is shut down, which
would be a leak for user code. With the new symbol-based
host-defined option we can just get the ModuleWrap from the
JS-land WeakMap so there's now no need to maintain this
strong reference. This would at least fix the leak for
vm.SyntheticModule. vm.SourceTextModule is still leaking
due to the strong persistent reference to the v8::Module.
PR-URL: #48510
Refs: #44211
Refs: #42080
Refs: #47096
Refs: #43205
Refs: #38695
Reviewed-By: Antoine du Hamel [email protected]
Reviewed-By: Benjamin Gruenbaum [email protected]
Reviewed-By: Stephen Belanger [email protected]
module: fix the leak in SourceTextModule and ContextifySript
Replace the persistent handles to v8::Module and
v8::UnboundScript with an internal reference that V8's GC is
aware of to fix the leaks.
PR-URL: #48510
Refs: #44211
Refs: #42080
Refs: #47096
Refs: #43205
Refs: #38695
Reviewed-By: Antoine du Hamel [email protected]
Reviewed-By: Benjamin Gruenbaum [email protected]
Reviewed-By: Stephen Belanger [email protected]
test: add checkIfCollectable to test/common/gc.js
PR-URL: #49671
Reviewed-By: James M Snell [email protected]
Reviewed-By: Benjamin Gruenbaum [email protected]
Reviewed-By: Yagiz Nizipli [email protected]
Reviewed-By: Michaël Zasso [email protected]
test: use checkIfCollectable in vm leak tests
Previously we simply create a lot of the target objects and check
if the process crash due to OOM. Due to how we use emphemeron GC
to handle memory management, which is inefficient but necessary
for correctness, the tests can produce false positives as
the GC isn't efficient enough to catch up with a very fast
heap growth.
This patch uses a new checkIfCollectable() utility to terminate the
test early once we detect that any of the target object can actually
be garbage collected. This should lower the chance of false positives.
As a drive-by this also allows us to use setImmediate() to grow the
heap even faster to make the tests run faster.
PR-URL: #49671
Reviewed-By: James M Snell [email protected]
Reviewed-By: Benjamin Gruenbaum [email protected]
Reviewed-By: Yagiz Nizipli [email protected]
Reviewed-By: Michaël Zasso [email protected]
test: deflake test-vm-contextified-script-leak
Similar to the test-vm-source-text-module-leak fix, use a snapshot
to force a thorough GC in order to prevent false positives.
PR-URL: #49710
Refs: nodejs/reliability#669
Reviewed-By: Franziska Hinkelmann [email protected]
Reviewed-By: Michaël Zasso [email protected]
Reviewed-By: Luigi Pinca [email protected]
Reviewed-By: Rich Trott [email protected]