Migrating the packaging system to `poetry` with `pyproject.toml` #409

fredgrub · 2023-10-29T14:06:34Z

Deleted old setup files requirements.txt, setup.cfg, setup.py, MANIFEST.in
Added poetry files poetry.toml, pyproject.toml, poetry.lock
Added .pyenv-version and .tool-versions for pyenv and asdf (useful for testing multiple python versions with tox)
Updated Makefile, CONTRIBUTING.rst, tox.ini

I tested the upload to PyPI using TestPyPI. The package is located at https://test.pypi.org/project/pipreqs-build-test/ and can be installed via pip. I installed it as a user and it worked correctly.

codecov-commenter · 2023-10-29T14:09:15Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (368e9ae) 90.07% compared to head (679004f) 90.07%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@           Coverage Diff           @@
##             next     #409   +/-   ##
=======================================
  Coverage   90.07%   90.07%           
=======================================
  Files           2        2           
  Lines         262      262           
=======================================
  Hits          236      236           
  Misses         26       26

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Fernando-crz

First of all, I had some problems running poetry install --with dev.
Got the error "[Errno 2] No such file or directory: 'python'". Seems to be an alias problem though, because I could fix the error running
sudo apt install python-is-python3
It seems like poetry doesn't support the python command being named "python3". Maybe we can fix it using some specific poetry config during installation?

Also, for some reason, I had some problems running the tox's tests in python version 3.11. Here's the errors that I got:

======================================================================
FAIL: test_get_use_local_only (tests.test_pipreqs.TestPipreqs.test_get_use_local_only)
Test without checking PyPI, check to see if names of local
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/fernando/home/labxp/pipreqsxp/pipreqs/tests/test_pipreqs.py", line 142, in test_get_use_local_only
    self.assertTrue(item["name"].lower() in self.local)
AssertionError: False is not true

======================================================================
FAIL: test_init (tests.test_pipreqs.TestPipreqs.test_init)
Test that all modules we will test upon are in requirements file
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/fernando/home/labxp/pipreqsxp/pipreqs/tests/test_pipreqs.py", line 169, in test_init
    self.assertEqual(data, sorted(data))
AssertionError: Lists differ: ['ana[186 chars]s==3.1.0', 'pyflakes==3.0.1', 'requests==2.31.[36 chars]4.0'] != ['ana[186 chars]s==3.0.1', 'pyflakes==3.1.0', 'requests==2.31.[36 chars]4.0']

First differing element 9:
'pyflakes==3.1.0'
'pyflakes==3.0.1'

  ['analytics_python==1.4.post1',
   'beautifulsoup4==4.12.2',
   'boto==2.49.0',
   'docopt==0.6.2',
   'flask==2.2.2',
   'flask_seasurf==1.1.1',
   'ipython==8.12.0',
   'nose==1.3.7',
   'peewee==3.17.0',
+  'pyflakes==3.0.1',
   'pyflakes==3.1.0',
-  'pyflakes==3.0.1',
   'requests==2.31.0',
   'sqlalchemy==1.4.39',
   'ujson==5.4.0']

======================================================================
FAIL: test_init_local_only (tests.test_pipreqs.TestPipreqs.test_init_local_only)
Test that items listed in requirements.text are the same
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/fernando/home/labxp/pipreqsxp/pipreqs/tests/test_pipreqs.py", line 195, in test_init_local_only
    self.assertTrue(item[0].lower() in self.local)
AssertionError: False is not true

----------------------------------------------------------------------

For some odd reason, those errors only occur running python 3.11, but work on other python versions. Kinda odd because the tests ran in github actions seem to work successfully.

CONTRIBUTING.rst

mateuslatrova · 2023-10-30T20:04:40Z

the correct name of the file with the python versions for pyenv is ".python-version"

mateuslatrova · 2023-10-30T20:06:16Z

First of all, I had some problems running poetry install --with dev. Got the error "[Errno 2] No such file or directory: 'python'". Seems to be an alias problem though, because I could fix the error running sudo apt install python-is-python3 It seems like poetry doesn't support the python command being named "python3". Maybe we can fix it using some specific poetry config during installation?

Also, for some reason, I had some problems running the tox's tests in python version 3.11. Here's the errors that I got:

======================================================================
FAIL: test_get_use_local_only (tests.test_pipreqs.TestPipreqs.test_get_use_local_only)
Test without checking PyPI, check to see if names of local
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/fernando/home/labxp/pipreqsxp/pipreqs/tests/test_pipreqs.py", line 142, in test_get_use_local_only
    self.assertTrue(item["name"].lower() in self.local)
AssertionError: False is not true

======================================================================
FAIL: test_init (tests.test_pipreqs.TestPipreqs.test_init)
Test that all modules we will test upon are in requirements file
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/fernando/home/labxp/pipreqsxp/pipreqs/tests/test_pipreqs.py", line 169, in test_init
    self.assertEqual(data, sorted(data))
AssertionError: Lists differ: ['ana[186 chars]s==3.1.0', 'pyflakes==3.0.1', 'requests==2.31.[36 chars]4.0'] != ['ana[186 chars]s==3.0.1', 'pyflakes==3.1.0', 'requests==2.31.[36 chars]4.0']

First differing element 9:
'pyflakes==3.1.0'
'pyflakes==3.0.1'

  ['analytics_python==1.4.post1',
   'beautifulsoup4==4.12.2',
   'boto==2.49.0',
   'docopt==0.6.2',
   'flask==2.2.2',
   'flask_seasurf==1.1.1',
   'ipython==8.12.0',
   'nose==1.3.7',
   'peewee==3.17.0',
+  'pyflakes==3.0.1',
   'pyflakes==3.1.0',
-  'pyflakes==3.0.1',
   'requests==2.31.0',
   'sqlalchemy==1.4.39',
   'ujson==5.4.0']

======================================================================
FAIL: test_init_local_only (tests.test_pipreqs.TestPipreqs.test_init_local_only)
Test that items listed in requirements.text are the same
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/fernando/home/labxp/pipreqsxp/pipreqs/tests/test_pipreqs.py", line 195, in test_init_local_only
    self.assertTrue(item[0].lower() in self.local)
AssertionError: False is not true

----------------------------------------------------------------------

For some odd reason, those errors only occur running python 3.11, but work on other python versions. Kinda odd because the tests ran in github actions seem to work successfully.

I have run in my computer and the installation worked normally

tox.ini

alan-barzilay

Great work @fredgrub and thank you very much @Fernando-crz and @mateuslatrova for your reviews!

I still haven' t tried to install it locally and test it, I have simply read the changes so I will be back later to let you know if anything goes wrong and if by any chance I can replicate the issue encountered by fernando

.python-version

.tool-versions

CONTRIBUTING.rst

poetry.toml

pyproject.toml

tox.ini

alan-barzilay · 2023-11-06T17:42:28Z

problem installing in a fresh venv from the test repo:

❯ python -m venv teste1
❯ source teste1/bin/activate
❯ pip install -i https://test.pypi.org/simple/ pipreqs-build-test
Looking in indexes: https://test.pypi.org/simple/
Collecting pipreqs-build-test
  Obtaining dependency information for pipreqs-build-test from https://test-files.pythonhosted.org/packages/07/34/71313e66821e8c291c322079e4110fe8e06aca6a97d3343b1e401a8be7f7/pipreqs_build_test-0.4.15-py3-none-any.whl.metadata
  Downloading https://test-files.pythonhosted.org/packages/07/34/71313e66821e8c291c322079e4110fe8e06aca6a97d3343b1e401a8be7f7/pipreqs_build_test-0.4.15-py3-none-any.whl.metadata (7.5 kB)
Collecting docopt==0.6.2 (from pipreqs-build-test)
  Downloading https://test-files.pythonhosted.org/packages/f9/d7/d8f8d3e2d51df48c181d8e3706396c6fc2759b120b6be06721f00d84fe94/docopt-0.6.2-py2.py3-none-any.whl (19 kB)
INFO: pip is looking at multiple versions of pipreqs-build-test to determine which version is compatible with other requirements. This could take a while.
Collecting pipreqs-build-test
  Obtaining dependency information for pipreqs-build-test from https://test-files.pythonhosted.org/packages/07/ea/b8f7be3ae887aaac15343b04a6dd7fec45ef1e1b7dc8dc66e5b093611957/pipreqs_build_test-0.4.14-py3-none-any.whl.metadata
  Downloading https://test-files.pythonhosted.org/packages/07/ea/b8f7be3ae887aaac15343b04a6dd7fec45ef1e1b7dc8dc66e5b093611957/pipreqs_build_test-0.4.14-py3-none-any.whl.metadata (7.5 kB)
  Obtaining dependency information for pipreqs-build-test from https://test-files.pythonhosted.org/packages/1f/e8/04e431dbfd6ee635c5fcf1d7cb0eb90a1af836ecdfb155e5f76a10278063/pipreqs_build_test-0.4.13.1-py3-none-any.whl.metadata
  Downloading https://test-files.pythonhosted.org/packages/1f/e8/04e431dbfd6ee635c5fcf1d7cb0eb90a1af836ecdfb155e5f76a10278063/pipreqs_build_test-0.4.13.1-py3-none-any.whl.metadata (7.5 kB)
ERROR: Cannot install pipreqs-build-test==0.4.13.1, pipreqs-build-test==0.4.14 and pipreqs-build-test==0.4.15 because these package versions have conflicting dependencies.

The conflict is caused by:
    pipreqs-build-test 0.4.15 depends on yarg==0.1.9
    pipreqs-build-test 0.4.14 depends on yarg==0.1.9
    pipreqs-build-test 0.4.13.1 depends on yarg==0.1.9

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

alan-barzilay · 2023-11-06T17:48:23Z

could not replicate the issue encountered by fernando, poetry install --with dev worked as intended for me

fredgrub · 2023-11-08T13:04:05Z

@alan-barzilay, answering your comment #409 (comment), there is a peculiarity regarding using the test-pypi repo. Only pipreqs-build-test is available from test-pypi, the other dependencies such as yarg aren't guaranteed to be there. So we should explicitly make it fetch the main pypi repo. This is the command to do that

pip install --extra-index-url https://pypi.org/simple -i https://test.pypi.org/simple/ pipreqs-build-test

- Deleted old setup files `requirements.txt`, `setup.cfg`, `setup.py`, `MANIFEST.in` - Added poetry files `poetry.toml`, `pyproject.toml`, `poetry.lock` - Added `.pyenv-version` and `.tool-versions` for `pyenv` and `asdf` - Updated `Makefile`, `CONTRIBUTING.rst`, `tox.ini`

alan-barzilay

@alan-barzilay, answering your comment #409 (comment), there is a peculiarity regarding using the test-pypi repo. Only pipreqs-build-test is available from test-pypi, the other dependencies such as yarg aren't guaranteed to be there. So we should explicitly make it fetch the main pypi repo. This is the command to do that

Great! I tried installing according to your instructions and everything worked as expected, I see no reason to not merge this. Thanks for all the hard work!

Fernando-crz suggested changes Oct 29, 2023

View reviewed changes

CONTRIBUTING.rst Outdated Show resolved Hide resolved

CONTRIBUTING.rst Show resolved Hide resolved

fredgrub force-pushed the build-system-update branch from 6eb555a to cf62146 Compare October 30, 2023 20:04

fredgrub force-pushed the build-system-update branch from cf62146 to 68ee3d7 Compare October 30, 2023 20:17

mateuslatrova reviewed Oct 31, 2023

View reviewed changes

tox.ini Show resolved Hide resolved

fredgrub mentioned this pull request Nov 1, 2023

Create GitHub Action to automatic publish to PyPI #411

Closed

alan-barzilay requested changes Nov 4, 2023

View reviewed changes

fredgrub force-pushed the build-system-update branch from 68ee3d7 to 6271ea7 Compare November 6, 2023 14:10

fredgrub requested a review from alan-barzilay November 6, 2023 14:12

fredgrub force-pushed the build-system-update branch 4 times, most recently from 2cc93da to 18f2032 Compare November 8, 2023 13:41

fredgrub force-pushed the build-system-update branch from 18f2032 to 679004f Compare November 8, 2023 13:42

alan-barzilay approved these changes Nov 8, 2023

View reviewed changes

alan-barzilay merged commit fb4560c into bndr:next Nov 8, 2023
7 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrating the packaging system to `poetry` with `pyproject.toml` #409

Migrating the packaging system to `poetry` with `pyproject.toml` #409

fredgrub commented Oct 29, 2023

codecov-commenter commented Oct 29, 2023 •

edited

Loading

Fernando-crz left a comment

mateuslatrova commented Oct 30, 2023

mateuslatrova commented Oct 30, 2023

alan-barzilay left a comment

alan-barzilay commented Nov 6, 2023

alan-barzilay commented Nov 6, 2023

fredgrub commented Nov 8, 2023

alan-barzilay left a comment

Migrating the packaging system to poetry with pyproject.toml #409

Migrating the packaging system to poetry with pyproject.toml #409

Conversation

fredgrub commented Oct 29, 2023

codecov-commenter commented Oct 29, 2023 • edited Loading

Codecov Report

Fernando-crz left a comment

Choose a reason for hiding this comment

mateuslatrova commented Oct 30, 2023

mateuslatrova commented Oct 30, 2023

alan-barzilay left a comment

Choose a reason for hiding this comment

alan-barzilay commented Nov 6, 2023

alan-barzilay commented Nov 6, 2023

fredgrub commented Nov 8, 2023

alan-barzilay left a comment

Choose a reason for hiding this comment

Migrating the packaging system to `poetry` with `pyproject.toml` #409

Migrating the packaging system to `poetry` with `pyproject.toml` #409

codecov-commenter commented Oct 29, 2023 •

edited

Loading