Questions or feature requests for use in unit testing

[jaydenseric]

Hi there! Thanks for working on this; it's great to see finally a Deno native package for working with a headless browser. Puppeteer has been very frustrating with Deno, and we're at the point now that it's not working at all. Attempting to use npm:[email protected] results type errors due to globals being set by npm Node.js types conflicting with globals set by Deno types. Attempting to use https://deno.land/x/[email protected] results in strange runtime errors now with modern Deno; and it's a huge number of major versions behind the latest Puppeteer version on npm. It seems to be unmaintained.

Consider this issue a question about how Astral can be used to achieve some things I previously had working with Puppeteer (when it was working with earlier Deno versions), and if there is no way, or an inconvenient way, consider it a feature request.

There are 2 main reasons I have been using headless browsers over the years:

1. Screenshotting web pages, or parts of pages. This is probably straightforward with your current Astral API, so I don't have questions about that right now.
2. Unit testing modules of a published library to check they function as intended in a browser. You want to be able to functionally create a browser page, do unit tests with it in an async callback, and then after it automatically does cleanup:
   • Code coverage is collected by the browser, and then combined with V8 code coverage data collected by Node.js/Deno when the same modules/functions were run directly in tests, to be able to have an accurate code coverage report of the library after the tests have run.
   • Any console messages emitted in the browser page are forwarded to the Node.js/Deno process and logged via console.group under a message indicating the messages came from the headless browser, using what API (warn, error, log, etc.).
   • Page errors are collected and after the tests callback completes, assert the array of page errors is empty. You don't want to end up in a situation where the unit tests are passing but there are uncaught page errors going on undetected.

How can Astral be used to achieve use-case number 2? Here is how it was achieved using Puppeteer:

https://github.com/jaydenseric/ruck/blob/aa068afa7f0630c4e3e8617b2936210c14815964/test/testPuppeteerPage.mjs

It was very difficult for me to work out at first, but that's just because the Puppeteer API doesn't provide a ergonomic ways to do it. Surely it would not be complicated to provide a specific API for this in Astral; an easy way to create a browser page for use in unit testing.

[lino-levan]

Thanks for the issue! You're right that at the moment, use-case number 1 is quite easy to fulfill. Astral could technically achieve use-case number 2 using the raw (but fully typed) celestial bindings but I totally agree that a more ergonomic API would be fitting to have. In an ideal world, how would you imagine such an API to work? I would love to work with you on designing something to fit your usecase.

[lowlighter]

I would also be interested into some way of collecting the code coverage from functions passed to page.evaluate() as they're currently left out by deno coverage despite being ran.

For 2.1, I feel like it'd be nice if we just had to pass something like newPage({coverage: "coverageDir"}) and it'll launch Profiler.startPreciseCoverage() and when page.close() it'd call Profiler.takePreciseCoverage() and Profiler.stopPreciseCoverage() and save the coverage to the specified directory automatically

For 2.2 and 2.3, I think it's pretty much #22 so maybe we could just have two more options console: "log" | "collect" | "null" (for Runtime.consoleAPICalled) and same for exceptions Runtime.exceptionThrown, where "log" would just log them to console, and "collect" would stores them into the page instance so they can be asserted or handled manually later on

[lino-levan]

I'm in favor of your solution to 2.1 and 2.2. I'm a little skeptical about 2.3, but I think I could be convinced.

[lowlighter]

I've just looked quickly for the coverage of Page.evaluate() functions

While it seems possible to map back to where the function originate from (using hacks on Error.prepareStackTrace to get back the caller filename/line number through v8 internals), unfortunately I don't think it's possible to merge it back with deno coverage, at least not without post-processing.

Thing is, if a coverage for a same module appears twice, deno seems to only take the first occurence so it'll still appears as uncovered, so you need to merge them (and coverage for deno is only written at the end of the tests). I feel unless there's a way to inject coverage status to deno's v8 it won't be possible to make straightforward way of doing it

Ideally it'd be nice also if there was a way to detect in deno if it's running in test mode and if coverage is enabled, so astral would be automatically able to transfer the coverage of function running in the browser and it'd just be as simple as using how to print coverage for deno currently. But for this I assume it requires some upstream changes, I don't think it's possible to do it seamlessly in userland

[lino-levan]

Completely agree with your vision here. I think this would be great but it does seam unfeasible at the current moment.