[code-infra] Update package layout for better ESM support

[Janpot]

closes #44055
closes #43980
closes #44265
closes #44180

Might close:

• #43242

Add a new mode to the build:

• Removes nested package.json files
• Stores ESM files with .mjs extension
• Add exports field to package.json
• Leave icon package layout as is, but add exports field. Next time we generate it we'll have to update the exports field
• Use rollup instead of babel to compile the modules. this makes it fully specify imports (made this optional for now)

Notes:

• added ./esm/* export to icons for people who used the vite alias workaround
• mjs extension and package.json exports are supported from node 12.17 onward
• Our ESM bundles don't work in Next.js due to

  ../../packages/mui-system/build/Box/index.mjs
  Error: It's currently unsupported to use "export *" in a client boundary. Please use named exports instead.
      at Object.transformSource (/Users/janpotoms/Projects/material-ui/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]_@[email protected]_babel-plugin-_fn2ktr3o4nwwaygw7yjiv6t4d4/node_modules/next/dist/build/webpack/loaders/next-flight-loader/index.js:87:31)
  
  Import trace for requested module:
  ../../packages/mui-system/build/Box/index.mjs
  ../../packages/mui-system/build/index.mjs
  ../../packages/mui-material/build/styles/index.mjs
  ./src/app/layout.tsx
  

  https://github.com/mui/material-ui/actions/runs/10578363192/job/29308312240?pr=43264#step:7:1960
• Another issue that is likely caused by our abismal ESM support [RSC] The entire @mui/joy/node bundle is included in the client bundle when Joy UI component is being imported into a RSC #37934

To Do:

• bundle size increase - explanation below
• make sure babel helpers are a dependency everywhere [code-infra] Add missing @babel/runtime dependency to @mui/material-pigment-css #43473 (comment)
• figure out the export * usage in files that have a 'use client' pragma. Blocked on [core] Remove 'use client' from index files and useAutocomplete reexport #41956
• investigate why it's resolving to cjs @mui/material-pigment-css
• copy license
• export modern target
• merge exports fields instead of overwrite
  • update @mui/material-pigment-css and @mui/icons-material
• document
• Confirmed that it fixes [ESM] @mui/icons-material move to pure ESM #35233 (comment)
• remove esmExternals from the docs (investigate) [docs] Remove esmExternals from the docs #43068
• X remove nested imports to @mui/material - [ButtonBase] Export types used in ButtonBase props #43530
• Check if the "closes" and "might close' issues are fixed

Bundle size increase

The bundle size increase can be fully attributed to the inclusion of the following ESM/CJS interop helper in the webpack output:

Code

/******/ 	/* webpack/runtime/create fake namespace object */
/******/ 	(() => {
/******/ 		var getProto = Object.getPrototypeOf ? (obj) => (Object.getPrototypeOf(obj)) : (obj) => (obj.__proto__);
/******/ 		var leafPrototypes;
/******/ 		// create a fake namespace object
/******/ 		// mode & 1: value is a module id, require it
/******/ 		// mode & 2: merge all properties of value into the ns
/******/ 		// mode & 4: return value when already ns object
/******/ 		// mode & 16: return value when it's Promise-like
/******/ 		// mode & 8|1: behave like require
/******/ 		__webpack_require__.t = function(value, mode) {
/******/ 			if(mode & 1) value = this(value);
/******/ 			if(mode & 8) return value;
/******/ 			if(typeof value === 'object' && value) {
/******/ 				if((mode & 4) && value.__esModule) return value;
/******/ 				if((mode & 16) && typeof value.then === 'function') return value;
/******/ 			}
/******/ 			var ns = Object.create(null);
/******/ 			__webpack_require__.r(ns);
/******/ 			var def = {};
/******/ 			leafPrototypes = leafPrototypes || [null, getProto({}), getProto([]), getProto(getProto)];
/******/ 			for(var current = mode & 2 && value; typeof current == 'object' && !~leafPrototypes.indexOf(current); current = getProto(current)) {
/******/ 				Object.getOwnPropertyNames(current).forEach((key) => (def[key] = () => (value[key])));
/******/ 			}
/******/ 			def['default'] = () => (value);
/******/ 			__webpack_require__.d(ns, def);
/******/ 			return ns;
/******/ 		};
/******/ 	})();

This is caused by the following import

material-ui/packages/mui-system/src/useMediaQuery/useMediaQuery.ts

Line 75 in eab1b9e

const maybeReactUseSyncExternalStore: undefined | any = (React as any)['useSyncExternalStore' + ''];

A potential switch to using use-sync-external-store has been explored in #43476 but ultimately it makes sense to keep it, it manifests in the webpack runtime.

Resolutions

To try out the changes use following resolutions:

"resolutions": {
    "@mui/internal-markdown": "https://pkg.csb.dev/mui/material-ui/commit/8a28484f/@mui/internal-markdown",
    "@mui/base": "https://pkg.csb.dev/mui/material-ui/commit/8a28484f/@mui/base",
    "@mui/codemod": "https://pkg.csb.dev/mui/material-ui/commit/8a28484f/@mui/codemod",
    "@mui/core-downloads-tracker": "https://pkg.csb.dev/mui/material-ui/commit/8a28484f/@mui/core-downloads-tracker",
    "@mui/docs": "https://pkg.csb.dev/mui/material-ui/commit/8a28484f/@mui/docs",
    "@mui/icons-material": "https://pkg.csb.dev/mui/material-ui/commit/8a28484f/@mui/icons-material",
    "@mui/lab": "https://pkg.csb.dev/mui/material-ui/commit/8a28484f/@mui/lab",
    "@mui/material-nextjs": "https://pkg.csb.dev/mui/material-ui/commit/8a28484f/@mui/material-nextjs",
    "@mui/material": "https://pkg.csb.dev/mui/material-ui/commit/8a28484f/@mui/material",
    "@mui/material-pigment-css": "https://pkg.csb.dev/mui/material-ui/commit/8a28484f/@mui/material-pigment-css",
    "@mui/private-theming": "https://pkg.csb.dev/mui/material-ui/commit/8a28484f/@mui/private-theming",
    "@mui/styled-engine-sc": "https://pkg.csb.dev/mui/material-ui/commit/8a28484f/@mui/styled-engine-sc",
    "@mui/styled-engine": "https://pkg.csb.dev/mui/material-ui/commit/8a28484f/@mui/styled-engine",
    "@mui/styles": "https://pkg.csb.dev/mui/material-ui/commit/8a28484f/@mui/styles",
    "@mui/system": "https://pkg.csb.dev/mui/material-ui/commit/8a28484f/@mui/system",
    "@mui/types": "https://pkg.csb.dev/mui/material-ui/commit/8a28484f/@mui/types",
    "@mui/utils": "https://pkg.csb.dev/mui/material-ui/commit/8a28484f/@mui/utils",
    "@mui/internal-babel-plugin-resolve-imports": "https://pkg.csb.dev/mui/material-ui/commit/8a28484f/@mui/internal-babel-plugin-resolve-imports",
    "@mui/internal-docs-utils": "https://pkg.csb.dev/mui/material-ui/commit/8a28484f/@mui/internal-docs-utils",
    "@mui/internal-scripts": "https://pkg.csb.dev/mui/material-ui/commit/8a28484f/@mui/internal-scripts",
    "@mui/internal-test-utils": "https://pkg.csb.dev/mui/material-ui/commit/8a28484f/@mui/internal-test-utils",
    "@mui/x-license": "https://pkg.csb.dev/mui/mui-x/commit/6e71f7c9/@mui/x-license",
    "@mui/x-data-grid": "https://pkg.csb.dev/mui/mui-x/commit/6e71f7c9/@mui/x-data-grid",
    "@mui/x-data-grid-pro": "https://pkg.csb.dev/mui/mui-x/commit/6e71f7c9/@mui/x-data-grid-pro",
    "@mui/x-data-grid-premium": "https://pkg.csb.dev/mui/mui-x/commit/6e71f7c9/@mui/x-data-grid-premium",
    "@mui/x-data-grid-generator": "https://pkg.csb.dev/mui/mui-x/commit/6e71f7c9/@mui/x-data-grid-generator",
    "@mui/x-date-pickers": "https://pkg.csb.dev/mui/mui-x/commit/6e71f7c9/@mui/x-date-pickers",
    "@mui/x-date-pickers-pro": "https://pkg.csb.dev/mui/mui-x/commit/6e71f7c9/@mui/x-date-pickers-pro",
    "@mui/x-charts": "https://pkg.csb.dev/mui/mui-x/commit/6e71f7c9/@mui/x-charts",
    "@mui/x-charts-pro": "https://pkg.csb.dev/mui/mui-x/commit/6e71f7c9/@mui/x-charts-pro",
    "@mui/x-charts-vendor": "https://pkg.csb.dev/mui/mui-x/commit/6e71f7c9/@mui/x-charts-vendor",
    "@mui/x-tree-view": "https://pkg.csb.dev/mui/mui-x/commit/6e71f7c9/@mui/x-tree-view",
    "@mui/x-tree-view-pro": "https://pkg.csb.dev/mui/mui-x/commit/6e71f7c9/@mui/x-tree-view-pro",
    "@mui/x-internals": "https://pkg.csb.dev/mui/mui-x/commit/6e71f7c9/@mui/x-internals"
  },

[mui-bot]

Netlify deploy preview

https://deploy-preview-43264--material-ui.netlify.app/

Menu: parsed: +1.74% , gzip: +2.03%
Unstable_NumberInput: parsed: +5.11% , gzip: +5.90%
@material-ui/core/useMediaQuery: parsed: +5.09% , gzip: +4.53%
MenuItem: parsed: +6.67% , gzip: +6.84%
Option: parsed: +6.97% , gzip: +7.39%
Select: parsed: +1.44% , gzip: +1.65%
Tab: parsed: +7.47% , gzip: +7.53%
TabPanel: parsed: +12.28% , gzip: +11.45%
useAutocomplete: parsed: +5.06% , gzip: +5.47%
@material-ui/unstyled: parsed: +0.43% , gzip: +0.44%
TablePagination: parsed: +7.54% , gzip: +8.23%
Tooltip: parsed: +0.53% , gzip: +0.45%
@mui/joy/Tooltip: parsed: +0.61% , gzip: +0.53%
@mui/joy/Menu: parsed: +0.49% , gzip: +0.46%
@mui/joy/Radio: parsed: +0.80% , gzip: +0.61%
@mui/joy/Checkbox: parsed: +0.78% , gzip: +0.59%
@mui/joy/Select: parsed: +0.46% , gzip: +0.43%
@mui/joy/Drawer: parsed: +0.65% , gzip: +0.54%
RadioGroup: parsed: +0.80% , gzip: +0.69%
@mui/joy/RadioGroup: parsed: +0.89% , gzip: +0.65%
and 23 more changes

Bundle size report

Details of bundle changes (Toolpad)
Details of bundle changes

Generated by 🚫 dangerJS against 8a28484