Make jest small

[styfle]

🚀 Feature Proposal

Make jest small again (less than 30 MB)

Motivation

It appears jest is the largest when comparing other test harnesses.

[email protected] was small...it looks like [email protected] is when the size exploded.
Now [email protected] is even larger.

Example

package	install size
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]

Click a badge above to see a history of the install size.

Pitch

Every repo needs to install jest to run it's own tests.

One repo with 50 MB is maybe not too bad but your app is split into 200 microservices, that's 10 GB of jest!

[wtgtybhertgeghgtwtg]

It's reporting differently for [email protected].

But, yeah, there are definitely some downstream dependencies that could be trimmed.

[SimenB]

Happy to trim deps if anyone is willing to figure out which we can drop without feature or performance regressions 🙂 Or if our .npmignores are not aggressive enough

[wtgtybhertgeghgtwtg]

There's not that much can be done on jest's side, just downstream stuff that's already been rejected (get handlebars to stop using uglify, get jsdom to stop using request, get people to stop bundling CLI's).
#6040 might help, though, but that's waiting on micromatch.

[SimenB]

The 46 mb jump was rough, but seems like a bug in counting. It's back to the level of 22, so maybe close this?

[styfle]

I updated the table in the original post above.

It looks like jest is still the largest so I think the issue is still relevant.

Update: also see jest dependency graph which might help identify unnecessary dependencies

[bugzpodder]

@styfle how do I add the package size badge in a github comment? Would like to use that elsewhere. ( facebook/create-react-app#3880 )

Also, when I just try to add the [email protected] it shows up as 61MB on my system.

> du -hd1 node_modules/ | sort -h
844K	node_modules//async
888K	node_modules//uglify-js
912K	node_modules//jest-util
924K	node_modules//source-map-support
972K	node_modules//babel-runtime
1.2M	node_modules//escodegen
1.4M	node_modules//node-notifier
1.8M	node_modules//cssstyle
2.2M	node_modules//ajv
2.9M	node_modules//jsdom
3.3M	node_modules//handlebars
3.8M	node_modules//fsevents
4.8M	node_modules//lodash
7.3M	node_modules//core-js
 61M	node_modules/
> cat package.json 
{
  "name": "j2301",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "dependencies": {
    "jest": "23.0.1"
  }
}

[styfle]

@bugzpodder If you click on the badge from the result page, it will display the markdown.

it shows up as 61MB on my system

Package Phobia is actually counting the raw bytes so it will be the smallest possible install size in a real world scenario. When you run npm install on your machine, there are many js files in a package that are smaller than the cluster/block size so they are actually increased to take up the full block.

For example, Windows default is 4 KB so a file smaller than 4 KB will take up 4 KB like this image.

Update: I created facebook/create-react-app#4534 so you can see the difference between that PR and this PR

[rickhanlonii]

It looks like jest is still the largest so I think the issue is still relevant.

I think Jest does more out of the box than the others listed, so is this is a fair expectation?

[styfle]

@rickhanlonii Good question!

I haven't used jest besides going through the first few sections of the docs which appears to be pretty common among these frameworks: asserts, matching, async, setup/teardown.

I think the mocking feature and snapshot feature are the only additional parts that are shipped with jest that are not shipped with other test frameworks.

Were those features added in [email protected] when the size spiked?

[SimenB]

We've got built-in support for babel transpilation, code coverage, fake timers, parallelisation across CPUs and really powerful VCS integration which combined with the dependency tree only runs the test related to files changed from a given commit, instead of the entire suite.

That last one (in addition to clear errors) is probably my favorite.

(and that's not even mentioning the watch mode (see https://github.com/jest-community/jest-watch-typeahead), and custom runners, allowing you to run e.g. eslint in the same watch process)

[thymikee]

We also ship JSDOM by default.

[styfle]

Very cool!

I didn't see VCS integration listed in the docs, it sounds glorious 🙌

Back to my original comment...

[email protected] was small...it looks like [email protected] is when the size exploded.

I see now that the CHANGELOG.md shows that this is the first version published to npm so I image the versions prior are another package with the same name.

That being said I still think there's room for improvement. But for now I will continue to use the competition to test micro services.

[qwertie]

I came to the issues page to say something similar: it's okay with me if Jest is huge-by-default, but couldn't there be a dramatically smaller jest-lite? And hey, do I really need Babel in my TypeScript project?

Or, if the Jest core APIs are designed to be drop-in compatible with some other API, I would be more comfortable using that other API until such time as I have a project even remotely close to 1MB of source code, which might justify having a 44MB "on disk" test framework with 418 non-jest dependencies.

[thymikee]

And hey, do I really need Babel in my TypeScript project

If you want to have mocking through jest.mock() working with import statements, you pretty much have to use it.

[qwertie]

I've never used mocking but if I did, I'd be happy to settle for the kind of mocking that doesn't parse my source code.

Based on size	I guess these don't.
TestDouble
Sinon

[thymikee]

Maybe it makes sense to make that configurable with allowJestMock or some other flag? cc @SimenB @cpojer @aaronabramov

[SimenB]

We need babel for code coverage as well.

I honestly don't understand why people are so averse to large dependencies in a server side dev only dependency

[cpojer]

I don't think this issue is actionable as it is. I do not care for the absolute size of Jest compared to other test runners. If we can provide a neat solution for people that's slightly bigger, I'm ok with that. It won't bloat production apps. If we can find low-hanging fruit and are able to remove a large chunk of our largest deps (20%+), I'm on board with that. I'm not willing to make any product changes in Jest to save a few bytes here and there though.

[qwertie]

Maybe "retain source-level compatibility with a much smaller test library" (jasmine?) is an actionable item?

Let the other library be the Preact to your React.

[wtgtybhertgeghgtwtg]

You're gonna run into a lot of these things even if you go to another test framework. If you pull in nyc for coverage, you're gonna get babel and handlebars and it ends up not being a whole lot smaller than jest.

I honestly don't understand why people are so averse to large dependencies in a server side dev only dependency

Complex dependency trees can present some problems other than just size. For example, all the work that went into getting micromatch@3 working with jest, since a dependency of a dependency of a dependency had a dynamic require. It's also a pain in a corporate environment, where a deep dependency can get flagged by a proxy. That nearly kept us from using create-react-app at work. It can also cause slowdown, like globby has been experiencing.

Also, it just makes me mad.

[milesj]

To continue with what @wtgtybhertgeghgtwtg said, it also has a lot to do with transitive dependencies, any conflicting versions, and possible stale/outdated versions. Large deps exacerbate this issue a lot of the time.

[qwertie]

@wtgtybhertgeghg I question the assumptions that (1) everyone wants a code coverage tool regardless of its cost and (2) that the tool should be installed per-repo rather than globally.

(Ideally (2) could be fixed in npm itself by having some sort of "recommended global dependency" feature where a project.json can ask for a big tool to be installed globally so that when you have many small repos they can share the same large tool, without completely losing dependency information as occurs today where --global tools are not listed in package.json at all. Oh well...)

[milesj]

IMO, I think the best approach would be to split jest up into a few more packages. Like jest-coverage which has istanbul as a peer dep, and allows us to configure/upgrade istanbul according to our app. The same could be said for jsdom and the other major deps, because as of right now, we have to wait for Jest to upgrade them in a major to pull in new changes.

Also has the added benefit of reducing size as consumers pick and choose what they want.

[Magnitus-]

Judging by this thread, I think it's clear that Jest is primarily a frontend test library with everything baked in.

For frontend, an integrated transpiler makes sense I guess (I really try to avoid it for my own projects, but every company I worked for uses one, I guess it is fashionable), but when you're trying to test some small model library in Node.js for the backend, suddenly all those things that are baked in for the frontend don't really make sense.

Call me paranoid, but from a security perspective, I worry when I see a seemingly unconstrained dependency tree I got to wade though in order to use a library (and in this case, the dependency tree is not really analytically tractable I'm afraid).

Overall, seems like a really nice full featured batteries-included frontend testing library though (just not really for me).

[styfle]

@Magnitus- Good point! Check out this spiderweb https://npm.anvaka.com/#/view/2d/jest/

[wtgtybhertgeghgtwtg]

Test transpilation actually doesn't account for that much of the dependency tree. A good chunk of the babel stuff is being used in other parts, like for inline snapshots or formatting messages. I think micromatch accounts for more dependencies here than babel does.

[TrySound]

I agree that big dependency tree makes node running slow. In some cases it's good to bundle dependencies like rollup does to achieve as fast as it possible start.

/cc @jonschlinkert

[wtgtybhertgeghgtwtg]

The problem with bundling is that it creates duplication. While it'll be smaller and less to require and parse if that library and only that library is using those bundled dependencies, it'll be larger and more to parse if something else that's being used does. While it may be an option for CLI's (prettier, rollup), it'd likely be a large net increase in size for users if jest tried to do something like that.

[ibezkrovnyi]

Disclaimer: I know that flat mode of organizing dependencies is most used,
I just wanted to notice that beside size of de-duplicated tree of dependencies there is also complexity for package manager to manipulate such huge graph of dependencies.

To check that I have added only [email protected] to pakage.json and did npm i --legacy-bundling:

	numbers
number of package.json files inside jest/node_modules	2163
size of jest folder	127,468,507 bytes

Did quick comparison to other packages we use (sorted):

package name	number of total dependencies before de-duplication
[email protected]	2163
[email protected]	673
[email protected]	896
@babel/[email protected]	515
[email protected]	300
(subjective) we also have package with a lot of dependencies related to webdriverio, selenium, etc	2133

Total number of dependencies before de-duplication we have in ui projects is over 10,000 (and similar number of added packages is reported by [email protected] after successful npm install).
package-lock.json size is ~5 mb

• npm dedupe already hangs
• npm install takes ages to complete

adding --max-old-space-size doesn't help

Unfortunately we can't change package manager, but even if yarn/pnpm is able to manage such dependency graphs much faster, even then complexity of keeping de-duplicated tree of dependencies is still there.

[probil]

@ibezkrovnyi Thanks for the idea regarding npm i --legacy-bundling.

I've just checked the latest version and seems like situation improved in jest@26 but not drastically

# jest v25.5.0
npm init --yes
npm i -D [email protected]
du -sh ./node_modules  

+ [email protected]
added 506 packages from 346 contributors and audited 506 packages in 18.644s
found 0 vulnerabilities

 53M	./node_modules
----------------

# jest v26.0.1
npm init --yes
npm i -D [email protected]
du -sh ./node_modules/

+ [email protected]
added 504 packages from 346 contributors and audited 504 packages in 20.415s
found 0 vulnerabilities

 50M	./node_modules/

Here is the list of "leaders":

u -shc ./node_modules/* | sort -rh | head -10
 50M	total
6.3M	./node_modules/node-notifier
5.4M	./node_modules/@babel
4.8M	./node_modules/lodash
3.6M	./node_modules/jsdom
1.2M	./node_modules/@types
1.1M	./node_modules/ajv
1.1M	./node_modules/acorn
996K	./node_modules/snapdragon
948K	./node_modules/@jest

[cpojer]

Let's remove node-notifier and make it an optional plugin. I don't know of anyone who uses it, and the people who do can install a separate package to hook it in. Does anyone want to work on a PR to make it its own package that will be used only when installed (and explicitly enabled)?

Note it is already an optional dependency (and at FB we blackhole it to an empty package).

[cpojer]

@SimenB is removing sane in #10048.

[SimenB]

Note it is already an optional dependency (and at FB we blackhole it to an empty package).

Yeah, it should just be a matter of moving it from optionalDependencies to devDependencies and tweaking the error message to tell people to install it. We do this for weak-napi already

[probil]

Here is approx. files count per dependency (top 10):

$ du -a | cut -d/ -f2 | sort | uniq -c | sort -nr | head -10
1051 lodash
 528 @babel
 485 jsdom
 217 @jest
 144 resolve
 144 @types
 126 cssstyle
 100 ajv
  99 node-notifier
  86 jest-snapshot

BTW, can't jsdom be made optional or peer? It's FE only, 13Mb in node_modules, uses deprecated [email protected]

[cpojer]

@probil JSDOM will become optional in Jest 28: https://jestjs.io/blog/2020/05/05/jest-26

Right now it's the default, so Jest 27 will make the node env the default, and then 28 will stop shipping it.

Note that for the monorepo at Facebook where we develop React Native, we blackhole jest-environment-jsdom to an empty module.

[probil]

@cpojer Cool! Thanks for update

[TrySound]

27 took so much time. How long to live with bloated node_modules and deprecation warnings until 28 is out? Maybe just kill jsdom in 27? Shouldn't be a big deal to add dependency for users.

[SimenB]

28 will come quite quick as I wanna drop node 10 and 15 as well. 27 should be just around the corner

[wopian]

Jest 27 has knocked off a further ~10 MB compared to 26 🎉

package	install size
[email protected]
[email protected]

[TrySound]

Removing jsdom in jest 28 will cut another 10
https://packagephobia.com/result?p=jsdom

[RDIL]

Hey all! I might be misunderstanding the exact reason, but it seems like installing Jest gets @jest/types installed too, which doesn't seem right... from my understanding, it is only for type definitions, so I don't quite know why it should be present (user-facing types live at @types/jest, right?)

[styfle]

Confirmed, Jest 28 reduced install size by 34%! Great job 🎉

package	install size
[email protected]
[email protected]

[SimenB]

@styfle awesome! Looking at the OP, this is the smallest Jest has been in "modern" times (i.e. when FB/Meta took over the jest package name - before that jest package wasn't this. 9d2b2bc). Would you say the issue is fixed?

FWIW, in the future we'll stop shipping babel-jest by default as well. Not sure how much smaller that'd make us as we use babel in snapshot tests, but still. Beyond that, I don't think there's much space saving to be had?

[styfle]

@SimenB Looks like babel-jest is nearly 11MB

If you remove that dependency then I think we can close this issue.

For comparison, here are the current competitors:

package	install size
[email protected]
[email protected]
[email protected]
[email protected]

[wtgtybhertgeghgtwtg]

@SimenB Looks like babel-jest is nearly 11MB

Most of that is from @jest/transform at 10.6MB, which comes in anyway (e.g. from @jest/core)

But, most of that seems to come from babel-plugin-istanbul at 7.30MB.

Drilling down, it's mostly @babel/core at 6.68MB.

[milesj]

I wonder if istanbul can be dropped entirely, since v8 code coveraga was released in nodejs v15.1.0, v14.18.0, v12.22.0.

[cpojer]

I'm in favor of dropping Istanbul.

[SimenB]

I'd think it makes sense for that to be part of dropping out of the box Babel support as we'd still pull in @babel/core from babel-jest.

We talked about babel-jest and its use cases in https://jestjs.io/blog/2020/05/05/jest-26, and we'd have to break all of them (out of the box) to get rid of @babel/core. Specifically

Once V8 coverage and native ESM support stabilizes in Jest, we will also be able remove babel-jest as a default but we will keep maintaining it.

I don't think anything has really changed here? V8 coverage is probably more viable, but ESM is still just as blocked as ever (although the mocks is mostly me having zero interest in working on it before ESM is more stable - others are free to jump in!)

That said, we can probably flip default coverage provider from babel to v8 and require the user to provide babel-plugin-istanbul. We'd still pull in @babel/core, so wouldn't really change anything, tho.

EDIT: Oh, and @jest/reporters pulls in istanbul-lib-instrument which is the dep that requires @babel/core, so that'd need a refactor as well. We could also of course drop support for babel coverage outside of babel-jest. It's used in the reporter to get coverage of uncovered files - could just as well be v8 coverage (we mark the entire file as uncovered - however that means e.g. type definitions are counted weirdly)

EDIT2: And as mentioned we use babel in inline snapshots, so that would need a refactor as well to yell at people that use inline snapshots to install babel

[RDIL]

@SimenB is making snapshots, say, an optional module (that manually needs to be installed) a possibility?

[SimenB]

we don't need babel for regular snapshots, so that seems overkill. Can do something for inline snapshots, tho.

That said, I personally do not care about the install size, so if anything is to happen here, somebody else has to do the work 🙂

[yifanwww]

Is there any reason that we need some @types/* deps listed in dependencies instead of devDependencies?

for example:

• babel-jest's dependencies
  • @types/babel__core
• babel-plugin-jest-hoist's dependencies
  • @types/babel__core
  • @types/babel__traverse
• jest-circus's dependencies
  • @types/node
• @jest/console's dependencies
  • @types/node
• @jest/core's dependencies
  • @types/node
• @jest/environment's dependencies
  • @types/node
• jest-environment-jsdom's dependencies
  • @types/jsdom
  • @types/node

[mixtur]

Who needs babel when there is esbuild. It won't make installation size much smaller, but at least dependency graph is less crazy that way.