Merge pull request #22 from abravalheri/master

Update `table` and add information about staggered migrations

README.md

@@ -23,10 +23,38 @@ markdown-friendly table.
 
 # Current status
 
-To see the status since the last time the scenarios were run open [table.md](table.md).
+To see the status since the last time the scenarios were run open [table.md](table.md)[^1].
 
 Please note:
-* Mixing package types within a single namespace is not supported. While it may work in some cases, it may also break depending on the software versions used, the install commands issued, or the order of commands. Don't mix types.
-* The `pkg_resources` method of namspacing is [no longer recommended](https://packaging.python.org/guides/packaging-namespace-packages/#pkg-resources-style-namespace-packages) and there is [the desire to deprecate it](https://github.com/pypa/python-packaging-user-guide/issues/265#issuecomment-290812581). It should only be used in legacy namespaces that already rely on it.
+* Mixing package types within a single namespace is not supported. While it may work in some cases, it may also break depending on the software versions used, the install commands issued, or the order of commands. It is generally advisable not to mix types.
+* The `pkg_resources` method of namspacing is [deprecated](https://setuptools.pypa.io/en/latest/pkg_resources.html).
+  Whenever possible, developers are encouraged to migrate away from it.
 * [PEP 420](https://www.python.org/dev/peps/pep-0420/) was accepted as part of Python 3.3. For wider compatibility (going back to Python 2.3), use the `pkgutil` method.
-* Zipped eggs don't play nicely with namespace packaging, and may be implicitly installed by commands like `python setup.py install`. To prevent this, it is recommended that you set [`zip_safe=False`](http://setuptools.readthedocs.io/en/latest/setuptools.html#setting-the-zip-safe-flag) in `setup.py`, as we do here.
+* Zipped eggs don't play nicely with namespace packaging, and may be implicitly installed by commands like `python setup.py install`. To prevent this, it is recommended that you set [`zip_safe=False`](http://setuptools.readthedocs.io/en/latest/setuptools.html#setting-the-zip-safe-flag) in `setup.py`, as we do here. Please also note that distributing packages via egg files is considered deprecated.
+* The tests reported in [table.md](table.md) use `pip` with *build isolation* and build-backend APIs.
+  This is triggered by the presence of a `pyproject.toml` file in each package source directory.
+  If your package does not have a `pyproject.toml` file,
+  `pip` might select a legacy (and deprecated) installation procedure, which can behave differently.
+
+# Remarks on staggered migrations
+
+It is difficult migrate away from deprecated `pkg_resources` namepaces in large projects.
+Ideally, all packages sharing a namespace should coordinate and simultaneously drop `__init__.py` files to conform with PEP 420.
+However, developers might be interested in carrying out a *staggered migration* plan and temporarily mix different namespacing techniques
+to mitigate the migration effort and spread the work load in time.
+
+Based on the results for the scenarios mixing `pkg_resources` and other namespace methods reported in
+[table.md](table.md) and [legacy_table.md](legacy_table.md),
+we can see that (in principle) a staggered migration plan can be successful,
+as long as the developers are willing to accept some limitations:
+
+- Deprecated installation methods will not be supported (e.g. `python setup.py install`),
+- Editable installations will not be supported.
+
+Please note, however, that these are preliminary studies.
+Developers should carry out an independent investigation, and check for the
+specific use cases they are interested in supporting.
+
+
+[^1]: If you would like to know about deprecated installation methods (e.g. via
+  `python setup.py install`) or Python 2.7, please have a look at [legacy_table.md](legacy_table.md).

legacy_table.md

@@ -0,0 +1,162 @@
+| Type | Interpreter | Package A command | Package B command | Status |
+| --- | --- | --- | --- | --- |
+| pkgutil | python2.7 | pip install . | pip install . | ✅ |
+| pkgutil | python2.7 | pip install . | pip install -e . | ✅ |
+| pkgutil | python2.7 | pip install . | python setup.py install | ✅ |
+| pkgutil | python2.7 | pip install . | python setup.py develop | ✅ |
+| pkgutil | python2.7 | pip install -e . | pip install . | ✅ |
+| pkgutil | python2.7 | pip install -e . | pip install -e . | ✅ |
+| pkgutil | python2.7 | pip install -e . | python setup.py install | ✅ |
+| pkgutil | python2.7 | pip install -e . | python setup.py develop | ✅ |
+| pkgutil | python2.7 | python setup.py install | pip install . | ✅ |
+| pkgutil | python2.7 | python setup.py install | pip install -e . | ✅ |
+| pkgutil | python2.7 | python setup.py install | python setup.py install | ✅ |
+| pkgutil | python2.7 | python setup.py install | python setup.py develop | ✅ |
+| pkgutil | python2.7 | python setup.py develop | pip install . | ✅ |
+| pkgutil | python2.7 | python setup.py develop | pip install -e . | ✅ |
+| pkgutil | python2.7 | python setup.py develop | python setup.py install | ✅ |
+| pkgutil | python2.7 | python setup.py develop | python setup.py develop | ✅ |
+| pkgutil | python3.7 | pip install . | pip install . | ✅ |
+| pkgutil | python3.7 | pip install . | pip install -e . | ✅ |
+| pkgutil | python3.7 | pip install . | python setup.py install | ✅ |
+| pkgutil | python3.7 | pip install . | python setup.py develop | ✅ |
+| pkgutil | python3.7 | pip install -e . | pip install . | ✅ |
+| pkgutil | python3.7 | pip install -e . | pip install -e . | ✅ |
+| pkgutil | python3.7 | pip install -e . | python setup.py install | ✅ |
+| pkgutil | python3.7 | pip install -e . | python setup.py develop | ✅ |
+| pkgutil | python3.7 | python setup.py install | pip install . | ✅ |
+| pkgutil | python3.7 | python setup.py install | pip install -e . | ✅ |
+| pkgutil | python3.7 | python setup.py install | python setup.py install | ✅ |
+| pkgutil | python3.7 | python setup.py install | python setup.py develop | ✅ |
+| pkgutil | python3.7 | python setup.py develop | pip install . | ✅ |
+| pkgutil | python3.7 | python setup.py develop | pip install -e . | ✅ |
+| pkgutil | python3.7 | python setup.py develop | python setup.py install | ✅ |
+| pkgutil | python3.7 | python setup.py develop | python setup.py develop | ✅ |
+| pkg_resources | python2.7 | pip install . | pip install . | ✅ |
+| pkg_resources | python2.7 | pip install . | pip install -e . | ✅ |
+| pkg_resources | python2.7 | pip install . | python setup.py install | ❌ |
+| pkg_resources | python2.7 | pip install . | python setup.py develop | ✅ |
+| pkg_resources | python2.7 | pip install -e . | pip install . | ✅ |
+| pkg_resources | python2.7 | pip install -e . | pip install -e . | ✅ |
+| pkg_resources | python2.7 | pip install -e . | python setup.py install | ❌ |
+| pkg_resources | python2.7 | pip install -e . | python setup.py develop | ✅ |
+| pkg_resources | python2.7 | python setup.py install | pip install . | ❌ |
+| pkg_resources | python2.7 | python setup.py install | pip install -e . | ❌ |
+| pkg_resources | python2.7 | python setup.py install | python setup.py install | ✅ |
+| pkg_resources | python2.7 | python setup.py install | python setup.py develop | ❌ |
+| pkg_resources | python2.7 | python setup.py develop | pip install . | ✅ |
+| pkg_resources | python2.7 | python setup.py develop | pip install -e . | ✅ |
+| pkg_resources | python2.7 | python setup.py develop | python setup.py install | ❌ |
+| pkg_resources | python2.7 | python setup.py develop | python setup.py develop | ✅ |
+| pkg_resources | python3.7 | pip install . | pip install . | ✅ |
+| pkg_resources | python3.7 | pip install . | pip install -e . | ✅ |
+| pkg_resources | python3.7 | pip install . | python setup.py install | ❌ |
+| pkg_resources | python3.7 | pip install . | python setup.py develop | ✅ |
+| pkg_resources | python3.7 | pip install -e . | pip install . | ✅ |
+| pkg_resources | python3.7 | pip install -e . | pip install -e . | ✅ |
+| pkg_resources | python3.7 | pip install -e . | python setup.py install | ❌ |
+| pkg_resources | python3.7 | pip install -e . | python setup.py develop | ✅ |
+| pkg_resources | python3.7 | python setup.py install | pip install . | ❌ |
+| pkg_resources | python3.7 | python setup.py install | pip install -e . | ❌ |
+| pkg_resources | python3.7 | python setup.py install | python setup.py install | ✅ |
+| pkg_resources | python3.7 | python setup.py install | python setup.py develop | ❌ |
+| pkg_resources | python3.7 | python setup.py develop | pip install . | ✅ |
+| pkg_resources | python3.7 | python setup.py develop | pip install -e . | ✅ |
+| pkg_resources | python3.7 | python setup.py develop | python setup.py install | ❌ |
+| pkg_resources | python3.7 | python setup.py develop | python setup.py develop | ✅ |
+| pep420 | python2.7 | pip install . | pip install . | ❌ |
+| pep420 | python2.7 | pip install . | pip install -e . | ❌ |
+| pep420 | python2.7 | pip install . | python setup.py install | ❌ |
+| pep420 | python2.7 | pip install . | python setup.py develop | ❌ |
+| pep420 | python2.7 | pip install -e . | pip install . | ❌ |
+| pep420 | python2.7 | pip install -e . | pip install -e . | ❌ |
+| pep420 | python2.7 | pip install -e . | python setup.py install | ❌ |
+| pep420 | python2.7 | pip install -e . | python setup.py develop | ❌ |
+| pep420 | python2.7 | python setup.py install | pip install . | ❌ |
+| pep420 | python2.7 | python setup.py install | pip install -e . | ❌ |
+| pep420 | python2.7 | python setup.py install | python setup.py install | ❌ |
+| pep420 | python2.7 | python setup.py install | python setup.py develop | ❌ |
+| pep420 | python2.7 | python setup.py develop | pip install . | ❌ |
+| pep420 | python2.7 | python setup.py develop | pip install -e . | ❌ |
+| pep420 | python2.7 | python setup.py develop | python setup.py install | ❌ |
+| pep420 | python2.7 | python setup.py develop | python setup.py develop | ❌ |
+| pep420 | python3.7 | pip install . | pip install . | ✅ |
+| pep420 | python3.7 | pip install . | pip install -e . | ✅ |
+| pep420 | python3.7 | pip install . | python setup.py install | ✅ |
+| pep420 | python3.7 | pip install . | python setup.py develop | ✅ |
+| pep420 | python3.7 | pip install -e . | pip install . | ✅ |
+| pep420 | python3.7 | pip install -e . | pip install -e . | ✅ |
+| pep420 | python3.7 | pip install -e . | python setup.py install | ✅ |
+| pep420 | python3.7 | pip install -e . | python setup.py develop | ✅ |
+| pep420 | python3.7 | python setup.py install | pip install . | ✅ |
+| pep420 | python3.7 | python setup.py install | pip install -e . | ✅ |
+| pep420 | python3.7 | python setup.py install | python setup.py install | ✅ |
+| pep420 | python3.7 | python setup.py install | python setup.py develop | ✅ |
+| pep420 | python3.7 | python setup.py develop | pip install . | ✅ |
+| pep420 | python3.7 | python setup.py develop | pip install -e . | ✅ |
+| pep420 | python3.7 | python setup.py develop | python setup.py install | ✅ |
+| pep420 | python3.7 | python setup.py develop | python setup.py develop | ✅ |
+| cross_pkg_resources_pkgutil | python2.7 | pip install . | pip install . | ✅ |
+| cross_pkg_resources_pkgutil | python2.7 | pip install . | pip install -e . | ❌ |
+| cross_pkg_resources_pkgutil | python2.7 | pip install . | python setup.py install | ❌ |
+| cross_pkg_resources_pkgutil | python2.7 | pip install . | python setup.py develop | ❌ |
+| cross_pkg_resources_pkgutil | python2.7 | pip install -e . | pip install . | ❌ |
+| cross_pkg_resources_pkgutil | python2.7 | pip install -e . | pip install -e . | ❌ |
+| cross_pkg_resources_pkgutil | python2.7 | pip install -e . | python setup.py install | ❌ |
+| cross_pkg_resources_pkgutil | python2.7 | pip install -e . | python setup.py develop | ❌ |
+| cross_pkg_resources_pkgutil | python2.7 | python setup.py install | pip install . | ✅ |
+| cross_pkg_resources_pkgutil | python2.7 | python setup.py install | pip install -e . | ✅ |
+| cross_pkg_resources_pkgutil | python2.7 | python setup.py install | python setup.py install | ✅ |
+| cross_pkg_resources_pkgutil | python2.7 | python setup.py install | python setup.py develop | ✅ |
+| cross_pkg_resources_pkgutil | python2.7 | python setup.py develop | pip install . | ❌ |
+| cross_pkg_resources_pkgutil | python2.7 | python setup.py develop | pip install -e . | ❌ |
+| cross_pkg_resources_pkgutil | python2.7 | python setup.py develop | python setup.py install | ❌ |
+| cross_pkg_resources_pkgutil | python2.7 | python setup.py develop | python setup.py develop | ❌ |
+| cross_pkg_resources_pkgutil | python3.7 | pip install . | pip install . | ✅ |
+| cross_pkg_resources_pkgutil | python3.7 | pip install . | pip install -e . | ❌ |
+| cross_pkg_resources_pkgutil | python3.7 | pip install . | python setup.py install | ❌ |
+| cross_pkg_resources_pkgutil | python3.7 | pip install . | python setup.py develop | ❌ |
+| cross_pkg_resources_pkgutil | python3.7 | pip install -e . | pip install . | ❌ |
+| cross_pkg_resources_pkgutil | python3.7 | pip install -e . | pip install -e . | ❌ |
+| cross_pkg_resources_pkgutil | python3.7 | pip install -e . | python setup.py install | ❌ |
+| cross_pkg_resources_pkgutil | python3.7 | pip install -e . | python setup.py develop | ❌ |
+| cross_pkg_resources_pkgutil | python3.7 | python setup.py install | pip install . | ✅ |
+| cross_pkg_resources_pkgutil | python3.7 | python setup.py install | pip install -e . | ✅ |
+| cross_pkg_resources_pkgutil | python3.7 | python setup.py install | python setup.py install | ✅ |
+| cross_pkg_resources_pkgutil | python3.7 | python setup.py install | python setup.py develop | ✅ |
+| cross_pkg_resources_pkgutil | python3.7 | python setup.py develop | pip install . | ❌ |
+| cross_pkg_resources_pkgutil | python3.7 | python setup.py develop | pip install -e . | ❌ |
+| cross_pkg_resources_pkgutil | python3.7 | python setup.py develop | python setup.py install | ❌ |
+| cross_pkg_resources_pkgutil | python3.7 | python setup.py develop | python setup.py develop | ❌ |
+| cross_pep420_pkgutil | python2.7 | pip install . | pip install . | ✅ |
+| cross_pep420_pkgutil | python2.7 | pip install . | pip install -e . | ❌ |
+| cross_pep420_pkgutil | python2.7 | pip install . | python setup.py install | ❌ |
+| cross_pep420_pkgutil | python2.7 | pip install . | python setup.py develop | ❌ |
+| cross_pep420_pkgutil | python2.7 | pip install -e . | pip install . | ❌ |
+| cross_pep420_pkgutil | python2.7 | pip install -e . | pip install -e . | ❌ |
+| cross_pep420_pkgutil | python2.7 | pip install -e . | python setup.py install | ❌ |
+| cross_pep420_pkgutil | python2.7 | pip install -e . | python setup.py develop | ❌ |
+| cross_pep420_pkgutil | python2.7 | python setup.py install | pip install . | ❌ |
+| cross_pep420_pkgutil | python2.7 | python setup.py install | pip install -e . | ❌ |
+| cross_pep420_pkgutil | python2.7 | python setup.py install | python setup.py install | ❌ |
+| cross_pep420_pkgutil | python2.7 | python setup.py install | python setup.py develop | ❌ |
+| cross_pep420_pkgutil | python2.7 | python setup.py develop | pip install . | ❌ |
+| cross_pep420_pkgutil | python2.7 | python setup.py develop | pip install -e . | ❌ |
+| cross_pep420_pkgutil | python2.7 | python setup.py develop | python setup.py install | ❌ |
+| cross_pep420_pkgutil | python2.7 | python setup.py develop | python setup.py develop | ❌ |
+| cross_pep420_pkgutil | python3.7 | pip install . | pip install . | ✅ |
+| cross_pep420_pkgutil | python3.7 | pip install . | pip install -e . | ✅ |
+| cross_pep420_pkgutil | python3.7 | pip install . | python setup.py install | ✅ |
+| cross_pep420_pkgutil | python3.7 | pip install . | python setup.py develop | ✅ |
+| cross_pep420_pkgutil | python3.7 | pip install -e . | pip install . | ✅ |
+| cross_pep420_pkgutil | python3.7 | pip install -e . | pip install -e . | ✅ |
+| cross_pep420_pkgutil | python3.7 | pip install -e . | python setup.py install | ✅ |
+| cross_pep420_pkgutil | python3.7 | pip install -e . | python setup.py develop | ✅ |
+| cross_pep420_pkgutil | python3.7 | python setup.py install | pip install . | ✅ |
+| cross_pep420_pkgutil | python3.7 | python setup.py install | pip install -e . | ✅ |
+| cross_pep420_pkgutil | python3.7 | python setup.py install | python setup.py install | ✅ |
+| cross_pep420_pkgutil | python3.7 | python setup.py install | python setup.py develop | ✅ |
+| cross_pep420_pkgutil | python3.7 | python setup.py develop | pip install . | ✅ |
+| cross_pep420_pkgutil | python3.7 | python setup.py develop | pip install -e . | ✅ |
+| cross_pep420_pkgutil | python3.7 | python setup.py develop | python setup.py install | ✅ |
+| cross_pep420_pkgutil | python3.7 | python setup.py develop | python setup.py develop | ✅ |

native/pkg_a/pyproject.toml

@@ -0,0 +1,3 @@
+[build-system]
+requires = ["setuptools"]
+build-backend = "setuptools.build_meta"

native/pkg_b/pyproject.toml

@@ -0,0 +1,3 @@
+[build-system]
+requires = ["setuptools"]
+build-backend = "setuptools.build_meta"

noxfile.py

@@ -29,20 +29,15 @@
 
 
 def install_packages(session, package_a, package_b, command_a, command_b):
-    session.install(
-        "--upgrade",
-        "setuptools",
-        "pip",
-        "wheel",
-        env={"PIP_CONSTRAINT": f"{HERE}/constraints.txt"},
-    )
+    env = {**os.environ, "PIP_CONSTRAINT": f"{HERE}/constraints.txt"}
+    session.install("--upgrade", "pip", env=env)
     session.chdir(package_a)
     session.run("rm", "-rf", "dist", "build", "*.egg-info")
-    session.run(*command_a)
+    session.run(*command_a, env=env)
     session.chdir(HERE)
     session.chdir(package_b)
     session.run("rm", "-rf", "dist", "build", "*.egg-info")
-    session.run(*command_b)
+    session.run(*command_b, env=env)
     session.chdir(HERE)
 
 
@@ -89,3 +84,23 @@ def session_cross_pkg_resources_pkgutil(session, command_a, command_b):
 def session_cross_pep420_pkgutil(session, command_a, command_b):
     install_packages(session, "native/pkg_a", "pkgutil/pkg_b", command_a, command_b)
     session.run("python", "verify_packages.py")
+
+
+@nox.session(python=USE_PYTHON_VERSIONS)
+@nox.parametrize('command_a', install_commands)
+@nox.parametrize('command_b', install_commands)
+def session_cross_pep420_pkg_resources(session, command_a, command_b):
+    install_packages(
+        session, 'native/pkg_a', 'pkg_resources/pkg_b', command_a, command_b
+    )
+    session.run("python", "verify_packages.py")
+
+
+@nox.session(python=USE_PYTHON_VERSIONS)
+@nox.parametrize('command_a', install_commands)
+@nox.parametrize('command_b', install_commands)
+def session_cross_pkg_resources_pep420(session, command_a, command_b):
+    install_packages(
+        session, 'pkg_resources/pkg_a', 'native/pkg_b', command_a, command_b
+    )
+    session.run("python", "verify_packages.py")

pkg_resources/pkg_a/pyproject.toml

@@ -0,0 +1,3 @@
+[build-system]
+requires = ["setuptools"]
+build-backend = "setuptools.build_meta"

pkg_resources/pkg_b/pyproject.toml

@@ -0,0 +1,3 @@
+[build-system]
+requires = ["setuptools"]
+build-backend = "setuptools.build_meta"

pkgutil/pkg_a/pyproject.toml

@@ -0,0 +1,3 @@
+[build-system]
+requires = ["setuptools"]
+build-backend = "setuptools.build_meta"

pkgutil/pkg_b/pyproject.toml

@@ -0,0 +1,3 @@
+[build-system]
+requires = ["setuptools"]
+build-backend = "setuptools.build_meta"