feat(napi/parser): introduce experimental MagicString

[Boshen]

Hold magic string instance on the Rust side for utf8 string manipulation.

[graphite-app]

Your org has enabled the Graphite merge queue for merging into main

Add the label “0-merge” to the PR and Graphite will automatically add it to the merge queue when it’s ready to merge. Or use the label “hotfix” to add to the merge queue as a hot fix.

You must have a Graphite account and log in to Graphite in order to use the merge queue. Sign up using this link.

[codspeed-hq]

CodSpeed Performance Report

Merging #7529 will not alter performance

_{Comparing napi-parser-magic-string (4bd3d10) with main (2e69720)}

Summary

✅ 29 untouched benchmarks

[yyx990803]

I like this - having this would actually allow Vue's compiler-sfc to replace @babel/parser and MagicString with just oxc-parser.

• Performance-wise, instead of making napi calls on every op, we should probably buffer the operations on the JS side and lazy-apply them only when toString is actually called.

• With the buffering, in Rolldown we can also expose this API (maybe with the AST already available via hook arguments) and allow plugin transform hooks to return an oxc ParserBuilder instance instead of strings so we don't even need to pass the finalized string to JS ever.

[overlookmotel]

I believe the API this PR provides is:

const code = 'const s: String = "测试"';
const oxc = new ParserBuilder(code);
const {program} = oxc.parseSync({ sourceFilename: 'test.ts' });
const stringLiteral = program.body[0].declarations[0].init;
// Magic string manipulation
oxc.remove(stringLiteral.start + 1, stringLiteral.end - 1);
// Get altered code
const alteredCode = oxc.toString();

How about returning the magic string from parseSync instead of passing in a ParserBuilder?

const code = 'const s: String = "测试"';
const {program, magicString} = parseSync(code, { sourceFilename: 'test.ts', magicString: true });
const stringLiteral = program.body[0].declarations[0].init;
// Magic string manipulation
magicString.remove(stringLiteral.start + 1, stringLiteral.end - 1);
// Get altered code
const alteredCode = magicString.toString();

i.e. You choose whether you want a MagicString to be returned from parseSync or not with an option, following same pattern as how you choose if you want a source map or not.

Builder patterns are quite rusty, but personally I think this is more idiomatic for a JS API.

[Boshen]

I believe the API this PR provides is:

const code = 'const s: String = "测试"';
const oxc = new ParserBuilder(code);
const {program} = oxc.parseSync({ sourceFilename: 'test.ts' });
const stringLiteral = program.body[0].declarations[0].init;
// Magic string manipulation
oxc.remove(stringLiteral.start + 1, stringLiteral.end - 1);
// Get altered code
const alteredCode = oxc.toString();

How about returning the magic string from parseSync instead of passing in a ParserBuilder?

const code = 'const s: String = "测试"';
const {program, magicString} = parseSync(code, { sourceFilename: 'test.ts', magicString: true });
const stringLiteral = program.body[0].declarations[0].init;
// Magic string manipulation
magicString.remove(stringLiteral.start + 1, stringLiteral.end - 1);
// Get altered code
const alteredCode = magicString.toString();

i.e. You choose whether you want a MagicString to be returned from parseSync or not with an option, following same pattern as how you choose if you want a source map or not.

Builder patterns are quite rusty, but personally I think this is more idiomatic for a JS API.

I can't make your suggestion work without cloning the string

#[napi]
pub struct ParserBuilder {
    cell: ParserBuilderImpl,
}

self_cell!(
    struct ParserBuilderImpl {
        owner: String,

        #[covariant]
        dependent: MagicString,
    }
);

MagicString is MagicString<'owner>

[overlookmotel]

Ah OK. I don't know the internals. If we could make my suggestion work, would you prefer it as an API?

[Boshen]

Merge activity

• Dec 10, 3:32 AM EST: The merge label '0-merge' was detected. This PR will be added to the Graphite merge queue once it meets the requirements.
• Dec 10, 3:33 AM EST: A user added this pull request to the Graphite merge queue.
• Dec 10, 3:37 AM EST: A user merged this pull request with the Graphite merge queue.