Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: use unusual chars in the path to ensure our tests are robust #48409

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open
12 changes: 10 additions & 2 deletions .github/workflows/test-linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ jobs:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
path: node
aduh95 marked this conversation as resolved.
Show resolved Hide resolved
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
with:
Expand All @@ -51,6 +52,13 @@ jobs:
- name: Environment Information
run: npx envinfo
- name: Build
run: make build-ci -j4 V=1 CONFIG_FLAGS="--error-on-warn"
run: make -C node build-ci -j4 V=1 CONFIG_FLAGS="--error-on-warn"
- name: Test
run: make run-ci -j4 V=1 TEST_CI_ARGS="-p actions --node-args='--test-reporter=spec' --node-args='--test-reporter-destination=stdout' --measure-flakiness 9"
run: make -C node run-ci -j4 V=1 TEST_CI_ARGS="-p actions --node-args='--test-reporter=spec' --node-args='--test-reporter-destination=stdout' --measure-flakiness 9"
- name: Re-run test in a folder whose name contains unusual chars
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it mean we'll double the time to run the tests in the GH Actions?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As you see in this PR, it adds 10 minutes

run: |
mv node "$DIR"
cd "$DIR"
./tools/test.py --flaky-tests keep_retrying -p actions -j 4
env:
DIR: dir%20with $unusual"chars?'åß∂ƒ©∆¬…`
14 changes: 11 additions & 3 deletions .github/workflows/test-macos.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ jobs:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
path: node
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
with:
Expand All @@ -64,7 +65,7 @@ jobs:
# happen anymore running this step here first, that's also useful
# information.)
- name: tools/doc/node_modules workaround
run: make tools/doc/node_modules
run: make -C node tools/doc/node_modules
# This is needed due to https://github.com/nodejs/build/issues/3878
- name: Cleanup
run: |
Expand All @@ -80,8 +81,15 @@ jobs:
df -h
echo "::endgroup::"
- name: Build
run: make build-ci -j$(getconf _NPROCESSORS_ONLN) V=1 CONFIG_FLAGS="--error-on-warn"
run: make -C node build-ci -j$(getconf _NPROCESSORS_ONLN) V=1 CONFIG_FLAGS="--error-on-warn"
- name: Free Space After Build
run: df -h
- name: Test
run: make run-ci -j$(getconf _NPROCESSORS_ONLN) V=1 TEST_CI_ARGS="-p actions --node-args='--test-reporter=spec' --node-args='--test-reporter-destination=stdout' --measure-flakiness 9"
run: make -C node run-ci -j$(getconf _NPROCESSORS_ONLN) V=1 TEST_CI_ARGS="-p actions --node-args='--test-reporter=spec' --node-args='--test-reporter-destination=stdout' --measure-flakiness 9"
- name: Re-run test in a folder whose name contains unusual chars
run: |
mv node "$DIR"
cd "$DIR"
./tools/test.py --flaky-tests keep_retrying -p actions -j 4
env:
DIR: dir%20with $unusual"chars?'åß∂ƒ©∆¬…`
21 changes: 11 additions & 10 deletions benchmark/fixtures/simple-error-stack.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 7 additions & 5 deletions benchmark/fixtures/simple-error-stack.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,13 @@
const lorem = 'Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.';

function simpleErrorStack() {
try {
(lorem as any).BANG();
} catch (e) {
return e.stack;
}
[1].map(() => {
try {
(lorem as any).BANG();
} catch (e) {
return e.stack;
}
})
}

export {
Expand Down
10 changes: 8 additions & 2 deletions lib/internal/source_map/prepare_stack_trace.js
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,15 @@ function prepareStackTraceWithSourceMaps(error, trace) {
const sm = fileName === lastFileName ?
lastSourceMap :
findSourceMap(fileName);
lastSourceMap = sm;
lastFileName = fileName;
// Only when a source map is found, cache it for the next iteration.
// This is a performance optimization to avoid interleaving with JS builtin function
// invalidating the cache.
// - at myFunc (file:///path/to/file.js:1:2)
// - at Array.map (<anonymous>)
// - at myFunc (file:///path/to/file.js:3:4)
if (sm) {
lastSourceMap = sm;
lastFileName = fileName;
return `${kStackLineAt}${serializeJSStackFrame(sm, callSite, trace[i + 1])}`;
}
} catch (err) {
Expand Down
44 changes: 34 additions & 10 deletions lib/internal/source_map/source_map_cache.js
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,9 @@ function maybeCacheSourceMap(filename, content, moduleInstance, isGeneratedSourc
}

const data = dataFromUrl(filename, sourceMapURL);
// `data` could be null if the source map is invalid.
// In this case, create a cache entry with null data with source url for test coverage.

const entry = {
__proto__: null,
lineLengths: lineLengths(content),
Expand Down Expand Up @@ -277,6 +280,8 @@ function sourceMapFromDataUrl(sourceURL, url) {
const parsedData = JSONParse(decodedData);
return sourcesToAbsolute(sourceURL, parsedData);
} catch (err) {
// TODO(legendecas): warn about invalid source map JSON string.
// But it could be verbose.
debug(err);
return null;
}
Expand Down Expand Up @@ -331,24 +336,43 @@ function sourceMapCacheToObject() {

/**
* Find a source map for a given actual source URL or path.
*
* This function may be invoked from user code or test runner, this must not throw
* any exceptions.
* @param {string} sourceURL - actual source URL or path
* @returns {import('internal/source_map/source_map').SourceMap | undefined} a source map or undefined if not found
*/
function findSourceMap(sourceURL) {
if (RegExpPrototypeExec(kLeadingProtocol, sourceURL) === null) {
sourceURL = pathToFileURL(sourceURL).href;
if (typeof sourceURL !== 'string') {
return undefined;
}
SourceMap ??= require('internal/source_map/source_map').SourceMap;
const entry = getModuleSourceMapCache().get(sourceURL) ?? generatedSourceMapCache.get(sourceURL);
if (entry === undefined) {

// No source maps for builtin modules.
if (sourceURL.startsWith('node:')) {
return undefined;
}
let sourceMap = entry.sourceMap;
if (sourceMap === undefined) {
sourceMap = new SourceMap(entry.data, { lineLengths: entry.lineLengths });
entry.sourceMap = sourceMap;

SourceMap ??= require('internal/source_map/source_map').SourceMap;
try {
if (RegExpPrototypeExec(kLeadingProtocol, sourceURL) === null) {
// If the sourceURL is an invalid path, this will throw an error.
sourceURL = pathToFileURL(sourceURL).href;
}
const entry = getModuleSourceMapCache().get(sourceURL) ?? generatedSourceMapCache.get(sourceURL);
if (entry?.data == null) {
return undefined;
}

let sourceMap = entry.sourceMap;
if (sourceMap === undefined) {
sourceMap = new SourceMap(entry.data, { lineLengths: entry.lineLengths });
entry.sourceMap = sourceMap;
}
return sourceMap;
} catch (err) {
debug(err);
return undefined;
}
return sourceMap;
}

module.exports = {
Expand Down
9 changes: 9 additions & 0 deletions test/common/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -1135,6 +1135,15 @@ const common = {
get checkoutEOL() {
return fs.readFileSync(__filename).includes('\r\n') ? '\r\n' : '\n';
},

get isInsideDirWithUnusualChars() {
return __dirname.includes('%') ||
(!isWindows && __dirname.includes('\\')) ||
__dirname.includes('$') ||
__dirname.includes('\n') ||
__dirname.includes('\r') ||
__dirname.includes('\t');
},
};

const validProperties = new Set(Object.keys(common));
Expand Down
2 changes: 2 additions & 0 deletions test/common/index.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ const {
isDumbTerminal,
isFreeBSD,
isIBMi,
isInsideDirWithUnusualChars,
isLinux,
isLinuxPPCBE,
isMainThread,
Expand Down Expand Up @@ -81,6 +82,7 @@ export {
isDumbTerminal,
isFreeBSD,
isIBMi,
isInsideDirWithUnusualChars,
isLinux,
isLinuxPPCBE,
isMainThread,
Expand Down
2 changes: 1 addition & 1 deletion test/es-module/test-esm-invalid-pjson.js
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ describe('ESM: Package.json', { concurrency: !process.env.TEST_PARALLEL }, () =>
assert.ok(stderr.includes('code: \'ERR_INVALID_PACKAGE_CONFIG\''), stderr);
assert.ok(
stderr.includes(
`Invalid package config ${path.toNamespacedPath(invalidJson)} while importing "invalid-pjson" from ${entry}.`
`Invalid package config ${path.toNamespacedPath(invalidJson)} while importing "invalid-pjson" from ${path.toNamespacedPath(entry)}.`
),
stderr
);
Expand Down
3 changes: 2 additions & 1 deletion test/fixtures/snapshot/child-process-sync.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ const {
function spawn() {
const { spawnSync, execFileSync, execSync } = require('child_process');
spawnSync(process.execPath, [ __filename, 'spawnSync' ], { stdio: 'inherit' });
execSync(`"${process.execPath}" "${__filename}" "execSync"`, { stdio: 'inherit' });
if (!process.env.DIRNAME_CONTAINS_SHELL_UNSAFE_CHARS)
execSync(`"${process.execPath}" "${__filename}" "execSync"`, { stdio: 'inherit' });
execFileSync(process.execPath, [ __filename, 'execFileSync' ], { stdio: 'inherit' });
}

Expand Down
14 changes: 7 additions & 7 deletions test/parallel/test-fs-cp.mjs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { mustCall, mustNotMutateObjectDeep } from '../common/index.mjs';
import { mustCall, mustNotMutateObjectDeep, isInsideDirWithUnusualChars } from '../common/index.mjs';

import assert from 'assert';
import fs from 'fs';
Expand Down Expand Up @@ -264,7 +264,7 @@ function nextdir(dirname) {
}

// It throws error if parent directory of symlink in dest points to src.
{
if (!isInsideDirWithUnusualChars) {
const src = nextdir();
mkdirSync(join(src, 'a'), mustNotMutateObjectDeep({ recursive: true }));
const dest = nextdir();
Expand All @@ -279,7 +279,7 @@ function nextdir(dirname) {
}

// It throws error if attempt is made to copy directory to file.
{
if (!isInsideDirWithUnusualChars) {
const src = nextdir();
mkdirSync(src, mustNotMutateObjectDeep({ recursive: true }));
const dest = './test/fixtures/copy/kitchen-sink/README.md';
Expand Down Expand Up @@ -310,7 +310,7 @@ function nextdir(dirname) {


// It throws error if attempt is made to copy file to directory.
{
if (!isInsideDirWithUnusualChars) {
const src = './test/fixtures/copy/kitchen-sink/README.md';
const dest = nextdir();
mkdirSync(dest, mustNotMutateObjectDeep({ recursive: true }));
Expand Down Expand Up @@ -346,7 +346,7 @@ function nextdir(dirname) {

// It throws error if attempt is made to copy src to dest
// when src is parent directory of the parent of dest
{
if (!isInsideDirWithUnusualChars) {
const src = nextdir('a');
const destParent = nextdir('a/b');
const dest = nextdir('a/b/c');
Expand All @@ -370,7 +370,7 @@ function nextdir(dirname) {
}

// It throws an error if attempt is made to copy socket.
if (!isWindows) {
if (!isWindows && !isInsideDirWithUnusualChars) {
const src = nextdir();
mkdirSync(src);
const dest = nextdir();
Expand Down Expand Up @@ -738,7 +738,7 @@ if (!isWindows) {
}

// It returns an error if attempt is made to copy socket.
if (!isWindows) {
if (!isWindows && !isInsideDirWithUnusualChars) {
const src = nextdir();
mkdirSync(src);
const dest = nextdir();
Expand Down
6 changes: 4 additions & 2 deletions test/parallel/test-inspector-strip-types.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,10 @@ if (!process.config.variables.node_use_amaro) common.skip('Requires Amaro');
const { NodeInstance } = require('../common/inspector-helper.js');
const fixtures = require('../common/fixtures');
const assert = require('assert');
const { pathToFileURL } = require('url');

const scriptPath = fixtures.path('typescript/ts/test-typescript.ts');
const scriptURL = pathToFileURL(scriptPath);

async function runTest() {
const child = new NodeInstance(
Expand All @@ -30,10 +32,10 @@ async function runTest() {
const scriptParsed = await session.waitForNotification((notification) => {
if (notification.method !== 'Debugger.scriptParsed') return false;

return notification.params.url === scriptPath;
return notification.params.url === scriptPath || notification.params.url === scriptURL.href;
});
// Verify that the script has a sourceURL, hinting that it is a generated source.
assert(scriptParsed.params.hasSourceURL);
assert(scriptParsed.params.hasSourceURL || common.isInsideDirWithUnusualChars);

await session.waitForPauseOnStart();
await session.runToCompletion();
Expand Down
2 changes: 2 additions & 0 deletions test/parallel/test-npm-install.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
const common = require('../common');
if (!common.hasCrypto)
common.skip('missing crypto');
if (common.isInsideDirWithUnusualChars)
common.skip('npm does not support this install path');

const path = require('path');
const exec = require('child_process').exec;
Expand Down
18 changes: 8 additions & 10 deletions test/parallel/test-permission-sqlite-load-extension.js
Original file line number Diff line number Diff line change
@@ -1,18 +1,16 @@
'use strict';
const common = require('../common');
const assert = require('node:assert');
const childProcess = require('child_process');

const code = `const sqlite = require('node:sqlite');
const db = new sqlite.DatabaseSync(':memory:', { allowExtension: true });
db.loadExtension('nonexistent');`.replace(/\n/g, ' ');

childProcess.exec(
`${process.execPath} --permission -e "${code}"`,
{},
common.mustCall((err, _, stderr) => {
assert.strictEqual(err.code, 1);
assert.match(stderr, /Error: Cannot load SQLite extensions when the permission model is enabled/);
assert.match(stderr, /code: 'ERR_LOAD_SQLITE_EXTENSION'/);
})
);
common.spawnPromisified(
process.execPath,
['--permission', '--eval', code],
).then(common.mustCall(({ code, stderr }) => {
assert.match(stderr, /Error: Cannot load SQLite extensions when the permission model is enabled/);
assert.match(stderr, /code: 'ERR_LOAD_SQLITE_EXTENSION'/);
assert.strictEqual(code, 1);
}));
12 changes: 12 additions & 0 deletions test/parallel/test-runner-source-maps-invalid-json.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading