Test-prerequisites or howto setup the integrationtest?

[drgsr]

Hello everybody,
currently, I am trying to build podman-py on my systems (ubuntu 20.04lts or debian bullseye).
I succeed to run all the unit tests -
but the integration-tests fail to say at least 50-60% (depending on the last git pull up to 90).
Therefore my questions:

• Are the integration tests checked in master supposed to run
  or are they "just waiting for implentation"?

• If yes: What is the intended setup/prerequisite for the integration tests to operate successfully?

  E.g.:

  • run as user/superuser/ user in a dedicated group?
  • minimal podman version?

I did not find hints in that direction.

Thanks in advance,
Günter

[jwhonce]

The tests should run using make tests. The Makefile will do a pip install --user ... to create the needed environment.

• Podman 3.2.0 is needed, and stated in the Makefile but not enforced.
• There may be a network test or two that will fail when run as a regular user. As root, all tests should be successful.
  • If running as a regular user, you need to setup podman service to run as that user. Meaning, the tests cannot be run as a regular user against podman running as root.

Please let me know what issues you find and we can make testing and the code more robust.

[drgsr]

Hello jwhonce,

I've experimented in various settings (debian 10 & podman 3.0.1&python3 3.9.2 ; ubuntu 20.04lts & podman 3.1.2 & python3 3.8.5)
My focus is on "python3", which is essential to my project - and - due to company restrictions - I am interested in the debian 10 target.

What I did:

cd podman-py; pytest-3 podman/tests/unittest > result_unittest.txt ; pytest-3 podman/tests/integrationtest > result_integrationtest.txt

The outputs of the unit-test and the integration tests in the scenario "debian 10&podman 3.0.1" are attached for details.
result_integrationtest.txt
result_unittest.txt

In more detail:
Personally, I prefer to call the "pytest-3" directly, which works fine with the unit-test, and most of the integration tests (and show the same success/failure, than the "coverage" approach)
In this case, one can also run "sudo pytest-3", in case the privileges would not be sufficient.

Findings:
Experience based on pytest-3@debian (and also on ubuntu)

• The unit tests seem to work
• the integration tests show some "successrate", where some do not work. It might be, that this is due to the older podman version - interestingly enough, it seems that the debian 10/podman 3.0.1 to provide a higher successrate

Experiments with the makefile - especially performed on ubuntu (for debian, I did not repeat)

• The makefile is unfortunately not that helpful, since the python version is unclear (2.x or 3.x)
• Reason: most of the calls go to "python" or "pip", which maps to python2... (at least for debian/ubuntu), even though there is already effort visible to escape this python-versioning hell -> Variable $(python)
• I did not succeed with make test but make unittest and make integrationtest did the job (<- executed on ubuntu 20.04lts)
• The makefile itself is not compatible to a "python virtual environment", since in the virtual environment dislikes the setup install option "--user"
• in a virtual environment, one can use the "coverage" test command, as listed in the makefile, leaving out the "--user" option (or alternatively stay away from virtual environment... )
• One requires the package "openssh-server" to be installed to execute the integration tests
• (minor interest) python3 is complaining about syntactic deprecated stuff, which will vanish in python 3.10

Cheers,
Günter

PS: Experiments have been performed on the following podman-py version:
kuka@dev-gschreiber:~/projekte/3rdparty/podman-py (master)$ git log
commit e1b9349 (HEAD -> master, origin/master, origin/HEAD)
Merge: cd6d151 003423b
Author: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
Date: Wed May 5 18:07:31 2021 +0000

Merge #85

85: Introduce sphinx processing for html API web site r=mwhahaha a=jwhonce

* Add docs target to Makefile
* Started updating some docstrings
* Added support for connections from containers.conf
* Ported more uses of XDG_* envvar to xdg package
* Downgrade modules in requirements.txt as needed by RHEL/Fedora
  packaging.

Signed-off-by: Jhon Honce <[email protected]>

Co-authored-by: Jhon Honce <[email protected]>

[jwhonce]

Additional information, we have moved to using the tox tool for testing. You can read pyproject.toml and tox.ini to see how the tests are now run. Each PR is run against the different python versions to catch syntactic and library changes.