feat: support `noOverwriteGlobs` for templates based on react #1191

lmgyuan · 2024-04-16T16:49:26Z

Description

add noOverwriteGlobs parameters to methods saveContentToFile and saveRenderedReactContent
pass this.noOverwriteGlobs to saveContentToFile
check the name of file to be written against the noOverwriteGlobs to determine whether it should be actually written to the target directory
add additional comments to relevant codes in the codebase so they are easier for future maintainers and contributors to understand.

Related issue(s)

Fixes #1128

merge updates in asyncapi/generator master branch

…n/generator into fix-noOverwriteGlobs-1128

- give noOverwriteGlobs a default value of [] in saveContentToFile and saveRenderedReactContent

derberg

@lmgyuan it's looking very good, now we just need a proper test to make sure it works long term.

Maybe we can add a test to test/integration.test.js 🤔
You could use https://github.com/asyncapi/template-for-generator-templates react-based template as a base for test. Generator template installation is npm based, so all features are included, so installing from GitHub link, pointing to specific commit will work (yes, we should not point to master to have stable tests).

This template generates a number of files -> https://github.com/asyncapi/template-for-generator-templates/blob/master/template/schemas/schema.js#L13-L20

So your test could generate output with noOverwriteGlobs ignoring some of them, and test evaluation would be as simple reading output dir and making sure that some filenames are not there. That should be reliable enough

Thoughts?

lmgyuan · 2024-04-17T17:28:41Z

@derberg Thanks for your comments and guidance on adding a test to the test/integration.test.js! I agree that we could add a test so we can check whether noOverwriteGlobs is working when we use React Engine. However, I am a bit confused about how to use the template-for-generator-templates. You seem to suggest that I could use https://github.com/asyncapi/template-for-generator-templates react-based template as a base for test, so do you mean something like the following:

const generator = new Generator('https://github.com/asyncapi/template-for-generator-templates.git#328', outputDir, {
      forceWrite: true,
      noOverwriteGlobs: ['**/schema.js', '**/anotherFile.js'] // Specify files to ignore
    });

There is not a specific reason for picking "#328" as the commit that this test is based on. I just picked one out of all the commits there are. I can see from the codes that the template generates a template that will generate some files, but may I ask how I could give it parameters so its output contains instructions about specific files that I want it to generate? For example, if I want the output generator template to have something like

<File name={`noOverwrite_test.html`}>
        <SchemaFile schemaName={schemaName} schema={schema} />
      </File>

so after I use the generator template to generate files, I can check that it does not overwrite noOverwrite_test.html (with --noOverwriteGlobs nOverwrite_test as a command line argument). But how do I use the template-for-generator-templates to generate the output template with instructions on file name in the first place?

Currently, instead of using template-for-generator-template, I am thinking maybe the following codes could work as a test for noOverwriteGlobs feature as well:

it('should ignore specified files with noOverwriteGlobs', async () => {
    const outputDir = generateFolderName();
    // Manually create a file to test if it's not overwritten
    const testFilePath = path.join(outputDir, 'index.html');
    // eslint-disable-next-line sonarjs/no-duplicate-string
    writeFileSync(testFilePath, '<script>const initialContent = "This should not change";</script>');

    // based on the html-template documentation, the default output is index.html
    const generator = new Generator('@asyncapi/[email protected]', outputDir, {
      forceWrite: true,
      outFilename: 'index.html',
      noOverwriteGlobs: ['**/index.html']
    });

    generator.generateFromFile(dummySpecPath);
    
    // Read the file to confirm it was not overwritten
    const fileContent = readFileSync(testFilePath, 'utf8');
    expect(fileContent).toBe('<script>const initialContent = "This should not change";</script>');
  });

Let me know what you think! : )

derberg · 2024-04-18T08:50:12Z

You seem to suggest that I could use https://github.com/asyncapi/template-for-generator-templates react-based template as a base for test, so do you mean something like the following:

yup exactly, just use last commit from that repo, like this

https://github.com/asyncapi/template-for-generator-templates.git#f8353637ca6bc2b4eaa5514ceb3bd06fe145e9c6

but @asyncapi/[email protected] is good too, especially that we use it in other test cases. I like your test suggestion 💯 especially that even though 1 file is ignored, with '**/index.html' globs are still tested

go ahead with implementation please

lmgyuan · 2024-04-18T14:04:28Z

@derberg just update the implementation!

lib/renderer/react.js

test/integration.test.js

derberg · 2024-04-29T08:26:27Z

@lmgyuan you'll need to solve conflicts first before you amend your integration tests

add await

- don't reuse methods from generator for now

This reverts commit b3b29a1.

This reverts commit 3e7afe7.

This reverts commit 900faa9.

This reverts commit 55f9ae7.

This reverts commit eebca77.

- use spyOn instead of jest.fn() in integration.test.js - use regex (?!\\/loglevel\\.js$) in package.json

derberg · 2024-05-27T14:29:30Z

test/integration.test.js

    return path.resolve(mainTestResultPath, crypto.randomBytes(4).toString('hex'));
  };

  jest.setTimeout(60000);
  const testOutputFile = 'test-file.md';

+  beforeAll(() => {
+    if (!existsSync(path.join(reactTemplate, 'package.json'))) {


why we need this?

Deleted. I used it during development because I could not find the package.json sometimes.

derberg · 2024-05-27T14:36:11Z

test/integration.test.js

@@ -55,4 +68,39 @@ describe('Integration testing generateFromFile() to make sure the result of the
    const file = await readFile(path.join(outputDir, testOutputFile), 'utf8');
    expect(file).toMatchSnapshot();
  });
+
+  it('should ignore specified files with noOverwriteGlobs', async () => {
+    const logSpyDebug = jest.spyOn(log, 'debug').mockImplementation(() => {});


why not log.debug = jest.fn(); and then expect(log.debug), then you do not need to do mockRestore() imho

I tried that. jest.fn() does not work but spyOn() does. The former does not actually track the log.debug() calls in other files, but the latter does by creating a layer of observation around the existing methods.

derberg · 2024-05-27T14:41:37Z

test/integration.test.js

+    // Check if the files have been overwritten
+    await expect(fileContent).toBe(testContent);
+    // Check if the log debug message was printed
+    await expect(logSpyDebug).toHaveBeenCalledWith(logMessage.skipOverwrite(testFilePath));


expect is not async call, so no need for await

derberg · 2024-05-27T14:47:36Z

test/integration.test.js

+
+    const outputDir = generateFolderName();
+    // Manually create a file to test if it's not overwritten
+    if (!await exists(outputDir)) {


actually I don't get why we need to validate if it exists, if a line earlier it is created

const generateFolderName = () => { // you always want to generate to new directory to make sure test runs in clear environment // normalize the path for cross-platform compatibility // return path.normalize(path.resolve(mainTestResultPath, crypto.randomBytes(4).toString('hex'))); return path.resolve(mainTestResultPath, crypto.randomBytes(4).toString('hex')); };

If I understand it correctly, generateFolderName() does not actually generate the directory, but just creates and returns a path string. Thus I manually checks and creates it.

I see that in other tests you have written, you did not check and create the directory. Did you not have directory or path not found issue? If I comment out line 78-80, I have

ENOENT: no such file or directory, open '/Users/yuanyuan/workspace/generator/test/temp/integrationTestResult/6224bfdd/test-file.md'

Add console.log for all the spied method calls

delete unused existsSync

delete console.log() and use readFileSync

sonarqubecloud · 2024-06-11T20:58:56Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

sonarqubecloud · 2024-06-20T14:33:35Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

lmgyuan · 2024-08-10T22:18:05Z

continued and merged in #1234

lmgyuan added 3 commits April 16, 2024 12:41

Merge pull request #1 from asyncapi/master

fc81484

merge updates in asyncapi/generator master branch

fix noOverwriteGlobs issue asyncapi#1128

94c04b5

Merge branch 'fix-noOverwriteGlobs-1128' of https://github.com/lmgyua…

4bccad3

…n/generator into fix-noOverwriteGlobs-1128

lmgyuan requested review from derberg, magicmatatjahu, jonaslagoni and asyncapi-bot-eve as code owners April 16, 2024 16:49

lmgyuan changed the title ~~fix: Fix no overwrite globs #1128~~ fix: fix no overwrite globs #1128 Apr 16, 2024

give noOverwriteGlobs a default value of []

57ec2ee

- give noOverwriteGlobs a default value of [] in saveContentToFile and saveRenderedReactContent

lmgyuan mentioned this pull request Apr 16, 2024

noOverwriteGlobs will not work in majority of cases in templates using react #1128

Closed

derberg changed the title ~~fix: fix no overwrite globs #1128~~ feat: support noOverwriteGlobs for templates based on react Apr 17, 2024

derberg reviewed Apr 17, 2024

View reviewed changes

lmgyuan added 10 commits April 18, 2024 08:55

Update integration.test.js

8333600

handle "no such file/directory" error

d467476

handle output path error

4999885

edit package require

19e11df

edit generator parameter

c0b9ec8

resolved asynchronous issue

d147b9f

solve asynchronous in test

7dd724f

make sure read is executed before checking

1ff0f22

delete unused packages

98a1e71

suppress no-duplicate-string

e9f5bd0

Merge branch 'master' into fix-noOverwriteGlobs-1128

695e960

derberg reviewed Apr 22, 2024

View reviewed changes

lib/renderer/react.js Outdated Show resolved Hide resolved

add log.debug to replace console.log

3bb9e6f

derberg reviewed Apr 23, 2024

View reviewed changes

test/integration.test.js Show resolved Hide resolved

lmgyuan added 14 commits May 22, 2024 09:45

update filtersRegistry, generator, integration.test

55f9ae7

add await

check whether a file path should be overwritten

900faa9

fixed logging issue

3e7afe7

update generator.js and react.js

b3b29a1

- don't reuse methods from generator for now

Revert "update generator.js and react.js"

6854f6b

This reverts commit b3b29a1.

Update package.json

fbd41c8

Revert "fixed logging issue"

09e7882

This reverts commit 3e7afe7.

Revert "check whether a file path should be overwritten"

49c8952

This reverts commit 900faa9.

Revert "update filtersRegistry, generator, integration.test"

eebca77

This reverts commit 55f9ae7.

Reapply "update filtersRegistry, generator, integration.test"

2f3a3ef

This reverts commit eebca77.

update package.json & integration.test.js

f5a6251

- use spyOn instead of jest.fn() in integration.test.js - use regex (?!\\/loglevel\\.js$) in package.json

eliminated unnecessary comments and codes;

0385f26

normalize file path in integration test

be28dea

normalize testFilePath in integration.test

12ada05

derberg reviewed May 27, 2024

View reviewed changes

lmgyuan added 8 commits May 28, 2024 14:23

Update integration.test.js

06848d4

Add console.log for all the spied method calls

Update integration.test.js

1ec96a4

delete unused existsSync

Update integration.test.js

61d795a

delete console.log() and use readFileSync

add await to saveContentToFile

f5180b7

used fspromise

4305a9c

Merge branch 'master' into fix-noOverwriteGlobs-1128

ce41fe0

Merge branch 'asyncapi:master' into master

c741598

test again

2ee44db

derberg assigned lmgyuan Jun 19, 2024

lmgyuan added 2 commits June 25, 2024 10:46

delete unnecessary declarations in integration test

2d77e3c

Merge branch 'master' into fix-noOverwriteGlobs-1128

47ae19b

lmgyuan closed this Jul 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: support `noOverwriteGlobs` for templates based on react #1191

feat: support `noOverwriteGlobs` for templates based on react #1191

lmgyuan commented Apr 16, 2024 •

edited

Loading

derberg left a comment

lmgyuan commented Apr 17, 2024 •

edited

Loading

derberg commented Apr 18, 2024

lmgyuan commented Apr 18, 2024

derberg commented Apr 29, 2024

derberg May 27, 2024

lmgyuan May 28, 2024

derberg May 27, 2024

lmgyuan May 28, 2024

derberg May 27, 2024

derberg May 27, 2024

lmgyuan May 28, 2024

lmgyuan May 28, 2024

sonarqubecloud bot commented Jun 11, 2024

sonarqubecloud bot commented Jun 20, 2024

lmgyuan commented Aug 10, 2024

feat: support noOverwriteGlobs for templates based on react #1191

feat: support noOverwriteGlobs for templates based on react #1191

Conversation

lmgyuan commented Apr 16, 2024 • edited Loading

derberg left a comment

Choose a reason for hiding this comment

lmgyuan commented Apr 17, 2024 • edited Loading

derberg commented Apr 18, 2024

lmgyuan commented Apr 18, 2024

derberg commented Apr 29, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sonarqubecloud bot commented Jun 11, 2024

Quality Gate passed

sonarqubecloud bot commented Jun 20, 2024

Quality Gate passed

lmgyuan commented Aug 10, 2024

feat: support `noOverwriteGlobs` for templates based on react #1191

feat: support `noOverwriteGlobs` for templates based on react #1191

lmgyuan commented Apr 16, 2024 •

edited

Loading

lmgyuan commented Apr 17, 2024 •

edited

Loading