Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

access the imported packages cache like require.cache #45206

Closed
ganeshkbhat opened this issue Oct 27, 2022 · 8 comments
Closed

access the imported packages cache like require.cache #45206

ganeshkbhat opened this issue Oct 27, 2022 · 8 comments
Labels
doc Issues and PRs related to the documentations.

Comments

@ganeshkbhat
Copy link

ganeshkbhat commented Oct 27, 2022

Affected URL(s)

https://nodejs.org/api/esm.html#esm_no_require_cache

Description of the problem

How do i access the imported packages cache like require.cache?

I am trying to do something like this:

c.mjs

const tester = import("./test.js");
import * as fs from "fs";
import console from 'console';

// !!import.meta.cache is not an API as per docs - 
// https://github.com/nodejs/help/issues/2806
// How do i access the imported packages cache like require.cache
if (!!import.meta && !!import.meta.cache) {
    for (let p in import.meta.cache) {
        console.log(p);
        // console.log(trim(p));
    }
}

c.js

const tester = require("./test.js");

// current access of require.cache i am using
if (!!require.cache) {
    for (let p in require.cache) {
        console.log(p);
        // console.log(trim(p));
    }
}


function trim(p) {
    var re = /(.*?).js/;
    var basename = path.basename(p);
    var moduleName = re.exec(basename)[1];
    return [moduleName, p];
}

test.js

var path = require('path');
var c = require('fs');
var f = require('child_process');

This question comes after

https://stackoverflow.com/questions/74215997/how-do-i-do-console-log-in-esm-or-es6-file-without-having-a-package-json-in-the?noredirect=1#comment131031974_74215997

nodejs/help#2806 (comment)

nodejs/help#2806

Note:
I have read this:

I am aware of this docs - https://nodejs.org/api/esm.html#esm_no_require_cache

A simple hack is also fine.

@ganeshkbhat ganeshkbhat added the doc Issues and PRs related to the documentations. label Oct 27, 2022
@aduh95
Copy link
Contributor

aduh95 commented Oct 27, 2022

The ESM cache is not exposed, and I believe this is per spec.

A simple hack is also fine.

In this case, you may like the following:

hack-loader.mjs:

globalThis.loadedURLs = [];

export function load(url, context, next) {
  globalThis.loadedURLs.push(url);
  return next(url, context);
}

c.mjs

const tester = import("./test.js");
import * as fs from "fs";
import console from 'console';

if (globalThis.loadedURLs) {
    for (let p of globalThis.loadedURLs) {
        console.log(p);
        // console.log(trim(p));
    }
}

If you run node --experimental-modules ./hack-loader.mjs c.mjs, you should get the result you are looking for. Note that this is a very hacky solution, and it will stop working as soon as we manage to get the loaders off-thread, so definitely not a long term solution.

I'm going to close this as I believe there's no bug to fix, but feel free to continue the discussion and ask more questions.

@aduh95 aduh95 closed this as not planned Won't fix, can't repro, duplicate, stale Oct 27, 2022
@ganeshkbhat
Copy link
Author

ganeshkbhat commented Oct 30, 2022

i have one more question. while this is a hack, does this help in capturing runtime dynamic imports (modules) when using a loader like this. can i use this hack in my project?

i am currently using a require/ import regex to check number of import statements, the modules and objects being accessed. here is the code base i am using for regex access:

  • import object from "module" - /import(?:[\s.*]([\w*{}\n\r\t, ]+)[\s*]from)?[\s*](?:["'](.*[\w]+)["'])?/gm
  • import("module") - /import\((?:["'\s]*([\w*{}\n\r\t, ]+)\s*)?["'\s](.*([@\w_-]+))["'\s].*\);$/mg + /(?:import\('?"?)(.*?)(?:'?"?\))/
  • require("module") - /(?:require\('?"?)(.*?)(?:'?"?\))/

https://github.com/ganeshkbhat/get-imports/blob/main/index.js

do you think it is wise to add this hack in the docs? it may help a lot of people with unnecessary "..." across comments. like https://stackoverflow.com/questions/74216420/access-the-imported-packages-and-modules-cache-like-require-cache

@ganeshkbhat
Copy link
Author

ganeshkbhat commented Oct 30, 2022

however, on a second, i believe this is a real opportunity area of exposing the import modules cache like the require.cache along with the ast. i am wishing to make a ast proposal to tc39 for scripting environments line nodejs, bun, etc even if not exposed in browsers. #45158 (comment) . however, i wished the imports cache was also included to be exposed in the server scripting environment.

@ganeshkbhat
Copy link
Author

ganeshkbhat commented Oct 30, 2022

@aduh95
Copy link
Contributor

aduh95 commented Oct 30, 2022

i am currently using a require/ import regex to check number of import statements, the modules and objects being accessed. here is the code base i am using for regex access:

You are more likely to get better results using a JS parser, such as acorn, otherwise you would have false positive (e.g. import statements in strings, in comments, etc.) and if the spec changes the syntax you would have to rethink your regex.

however, on a second, i believe this is a real opportunity area of exposing the import modules cache like the require.cache along with the ast. i am wishing to make a ast proposal to tc39 for scripting environments line nodejs, bun, etc even if not exposed in browsers.

I don't see we could do that, AFAIK it's not something we have access to, making a request to the TC39 would indeed be the way to go, however I'd be surprised if it hasn't been discussed before.

@aduh95 Seems like the hack of loaders is not somehow working.

What is not working? What command are you using? What output do you see? What were you expecting to get instead?

@ganeshkbhat
Copy link
Author

ganeshkbhat commented Feb 26, 2023

I am looking at this. https://tc39.es/process-document/ what recommendations do you have for this. I probably need help to address the nodejs part of it. I would like if I have some subject matter exertise from you here.

@ganeshkbhat
Copy link
Author

ganeshkbhat commented Apr 4, 2023

@aduh95 The normal loader commands are node --experimental-loader=./loader.mjs filename.mjs and I was expecting the output from console.log("Url + globalThis.loadedURLs: ", url, globalThis.loadedURLs) and console.log("globalThis.loadedURLs: ", p) using the https://github.com/ganeshkbhat/get-isesm/blob/main/loader.mjs and https://github.com/ganeshkbhat/get-isesm/blob/main/demos/loader.test.mjs respectively.

@icetbr
Copy link

icetbr commented Jun 4, 2024

This landed on node 22: --experimental-require-module. It allows mocha --watch to work with ESM modules, which was broken feature due to require.cache.

Here is the person who found the "fix"
https://joyeecheung.github.io/blog/2024/03/18/require-esm-in-node-js/

The docs
https://nodejs.org/api/modules.html#loading-ecmascript-modules-using-require

Here is where I first saw the merge
#51977 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc Issues and PRs related to the documentations.
Projects
None yet
Development

No branches or pull requests

3 participants