[FR]: Can we allow invoking the bazel jest_test target at the Class level rather to handling the entire module/package as a single test?

[deepu-mungamuri94]

What is the current behavior?

Currently, all tests within a module/folder/package are treated as a single test, and when we execute the bazel test //package_name:jest_test, all individual test files under package_name are run as a single test. As a result, whatever the timeout/size is, it applies to the whole package rather than to each test file within that module.

Describe the feature

At the moment, if a package has, say, 5K tests, it will time out after a given length of time.
Even if we change the timeout/size to medium/large, it will eventually time out if we have a big amount of tests in the future.

Proposed solution: The bazel test rule should produce individual targets for each test file included inside it, and the timeout should be applied to those generated test file level targets.
Java is performing the same function here

In summary, the purpose of this FR is to execute jest_test at the test suite/class level when the rule is invoked at the package level.
When the 'bazel test //package-name:jest_test' command is executed, it should build suite level targets within and run them in a bazel manner while adhering to timeout/size criteria. This manner, no matter how many tests are in the package, there are no TIMEOUTs.

[jbedard]

How would you configure it outside bazel? I assume the same could be done with jest_test...

[deepu-mungamuri94]

Thank you for your prompt answer, @jbedard.

We used the same configurations as that of the outsize bazel. The main difference is that we don't have time limits per package outside of bazel (direct invocation of the Jest framework). However, with Bazel, we now have time limits per rule. So, if the rule can be published per test suite/file rather than per package, we'll never have timeouts in Bazel because the test suite will never take that much amount of time.

I don't notice any issues with invocation, but we do have an issue with overall time considerations for the package. It would be ideal if the time constraint was applied to each test suite/class inside that package.

Similar implementations for Java may be found here. i.e., rule per test suite, despite the fact that it was activated at the package level.

[jbedard]

I think you can do the same as java with a macro that simply loops over your tests (or groups of tests such as per directory etc) and only includes those select test(s) in the jest_test(data)?

There's also the jest_test(shard_count) to make tests within a single target more concurrent, although that will only help if you're not already maximizing concurrency across targets.

[deepu-mungamuri94]

jest_test(shard_count) - Its not an option for us, as we already have maximum concurrency.

I think you can do the same as java with a macro that simply loops over your tests (or groups of tests such as per directory etc) and only includes those select test(s) in the jest_test(data)?
I can loop over the tests, and include single test suite/file inside data attribute of jest_test(data). This guarantees that the bazel test will be executed at file level within that package/module. But triggering that inside the loop makes the output cache data overrides with previous iteration output. So by end of the loop, we only have last test suite/file results inside test.log as it overrides at every iteration of the loop.

[jbedard]

Bazel will account for the shard_count, I believe a shart counts as a job and bazel limits the number of jobs. It might not help in your case if you've already maximized concurrency though.

[jbedard]

So by end of the loop, we only have last test suite/file results inside test.log as it overrides at every iteration of the loop.

Does the java test_suite not do the same thing? It just creates a java_test rule per file: