Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

7 tests fail #266

Open
yurivict opened this issue Apr 29, 2024 · 6 comments
Open

7 tests fail #266

yurivict opened this issue Apr 29, 2024 · 6 comments

Comments

@yurivict
Copy link

cd /usr/ports/devel/py-line-profiler/work-py39/line_profiler-4.1.3 && /usr/bin/env -i HOME=/usr/ports/devel/py-line-profiler/work-py39  PWD="${PWD}"  __MAKE_CONF=/nonexistent OSVERSION=1400509 PATH=/usr/local/libexec/ccache:/usr/ports/devel/py-line-profiler/work-py39/.bin:/home/yuri/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin TERM=xterm-256color XDG_DATA_HOME=/usr/ports/devel/py-line-profiler/work-py39  XDG_CONFIG_HOME=/usr/ports/devel/py-line-profiler/work-py39  XDG_CACHE_HOME=/usr/ports/devel/py-line-profiler/work-py39/.cache  HOME=/usr/ports/devel/py-line-profiler/work-py39 PATH=/usr/local/libexec/ccache:/usr/ports/devel/py-line-profiler/work-py39/.bin:/home/yuri/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin PKG_CONFIG_LIBDIR=/usr/ports/devel/py-line-profiler/work-py39/.pkgconfig:/usr/local/libdata/pkgconfig:/usr/local/share/pkgconfig:/usr/libdata/pkgconfig MK_DEBUG_FILES=no MK_KERNEL_SYMBOLS=no SHELL=/bin/sh NO_LINT=YES LDSHARED="cc -shared" PYTHONDONTWRITEBYTECODE= PYTHONOPTIMIZE= PREFIX=/usr/local  LOCALBASE=/usr/local  CC="cc" CFLAGS="-O2 -pipe  -fstack-protector-strong -fno-strict-aliasing "  CPP="cpp" CPPFLAGS=""  LDFLAGS=" -fstack-protector-strong " LIBS=""  CXX="c++" CXXFLAGS="-O2 -pipe -fstack-protector-strong -fno-strict-aliasing  " CCACHE_DIR="/tmp/.ccache" BSD_INSTALL_PROGRAM="install  -s -m 555"  BSD_INSTALL_LIB="install  -s -m 0644"  BSD_INSTALL_SCRIPT="install  -m 555"  BSD_INSTALL_DATA="install  -m 0644"  BSD_INSTALL_MAN="install  -m 444" PYTHONPATH=/usr/ports/devel/py-line-profiler/work-py39/stage/usr/local/lib/python3.9/site-packages /usr/local/bin/python3.9 -m pytest -k '' -rs -v -o addopts= 
=================================================================================== test session starts ====================================================================================
platform freebsd14 -- Python 3.9.18, pytest-7.4.4, pluggy-1.4.0 -- /usr/local/bin/python3.9
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase(PosixPath('/usr/ports/devel/py-line-profiler/work-py39/line_profiler-4.1.3/.hypothesis/examples'))
rootdir: /usr/ports/devel/py-line-profiler/work-py39/line_profiler-4.1.3
configfile: pyproject.toml
plugins: anyio-4.3.0, checkdocs-2.12.0, xdoctest-1.1.3, typeguard-4.2.1, hypothesis-6.98.18, enabler-3.1.1, cov-4.1.0, xdist-3.5.0
4 workers [29 items]   error                                                                                                                                                               
scheduling tests via LoadScheduling

::project 
tests/test_autoprofile.py::test_duplicate_function_autoprofile 
tests/test_autoprofile.py::test_single_function_autoprofile 
tests/test_autoprofile.py::test_autoprofile_module 
[gw1] [  3%] PASSED tests/test_autoprofile.py::test_single_function_autoprofile 
tests/test_autoprofile.py::test_multi_function_autoprofile 
[gw3] [  6%] PASSED tests/test_autoprofile.py::test_autoprofile_module 
tests/test_autoprofile.py::test_autoprofile_module_list 
[gw2] [ 10%] PASSED tests/test_autoprofile.py::test_duplicate_function_autoprofile 
tests/test_autoprofile.py::test_autoprofile_script_with_module 
[gw1] [ 13%] PASSED tests/test_autoprofile.py::test_multi_function_autoprofile 
tests/test_autoprofile.py::test_autoprofile_module_with_prof_imports 
[gw3] [ 17%] PASSED tests/test_autoprofile.py::test_autoprofile_module_list 
tests/test_autoprofile.py::test_autoprofile_script_with_prof_imports 
[gw2] [ 20%] PASSED tests/test_autoprofile.py::test_autoprofile_script_with_module 
tests/test_cli.py::test_cli 
[gw2] [ 24%] FAILED tests/test_cli.py::test_cli 
tests/test_complex_case.py::test_varied_complex_invocations 
[gw1] [ 27%] PASSED tests/test_autoprofile.py::test_autoprofile_module_with_prof_imports 
tests/test_cli.py::test_version_agreement 
[gw3] [ 31%] PASSED tests/test_autoprofile.py::test_autoprofile_script_with_prof_imports 
tests/test_complex_case.py::test_complex_example_python_none 
[gw3] [ 34%] PASSED tests/test_complex_case.py::test_complex_example_python_none 
tests/test_explicit_profile.py::test_explicit_profile_with_nothing 
[gw1] [ 37%] FAILED tests/test_cli.py::test_version_agreement 
tests/test_explicit_profile.py::test_simple_explicit_nonglobal_usage 
[gw1] [ 41%] FAILED tests/test_explicit_profile.py::test_simple_explicit_nonglobal_usage 
tests/test_explicit_profile.py::test_explicit_profile_with_environ_off 
[gw3] [ 44%] PASSED tests/test_explicit_profile.py::test_explicit_profile_with_nothing 
tests/test_explicit_profile.py::test_explicit_profile_with_environ_on 
[gw1] [ 48%] PASSED tests/test_explicit_profile.py::test_explicit_profile_with_environ_off 
tests/test_explicit_profile.py::test_explicit_profile_with_cmdline 
[gw3] [ 51%] PASSED tests/test_explicit_profile.py::test_explicit_profile_with_environ_on 
tests/test_explicit_profile.py::test_explicit_profile_with_kernprof 
[gw1] [ 55%] PASSED tests/test_explicit_profile.py::test_explicit_profile_with_cmdline 
tests/test_explicit_profile.py::test_explicit_profile_with_in_code_enable 
[gw3] [ 58%] PASSED tests/test_explicit_profile.py::test_explicit_profile_with_kernprof 
tests/test_explicit_profile.py::test_explicit_profile_with_duplicate_functions 
[gw1] [ 62%] PASSED tests/test_explicit_profile.py::test_explicit_profile_with_in_code_enable 
tests/test_import.py::test_import 
[gw3] [ 65%] PASSED tests/test_explicit_profile.py::test_explicit_profile_with_duplicate_functions 
tests/test_import.py::test_version 
[gw1] [ 68%] FAILED tests/test_import.py::test_import 
tests/test_ipython.py::TestIPython::test_init 
[gw3] [ 72%] FAILED tests/test_import.py::test_version 
tests/test_kernprof.py::TestKernprof::test_enable_disable 
[gw3] [ 75%] PASSED tests/test_kernprof.py::TestKernprof::test_enable_disable 
tests/test_kernprof.py::TestKernprof::test_gen_decorator 
[gw3] [ 79%] PASSED tests/test_kernprof.py::TestKernprof::test_gen_decorator 
[gw1] [ 82%] FAILED tests/test_ipython.py::TestIPython::test_init 
tests/test_kernprof.py::TestKernprof::test_function_decorator 
[gw1] [ 86%] PASSED tests/test_kernprof.py::TestKernprof::test_function_decorator 
[gw2] [ 89%] PASSED tests/test_complex_case.py::test_varied_complex_invocations 
tests/test_duplicate_functions.py::test_duplicate_function 
[gw2] [ 93%] FAILED tests/test_duplicate_functions.py::test_duplicate_function 
[gw0] [ 96%] PASSED ::project 
tests/test_assumptions.py::test_assumed_noop 
[gw0] [100%] PASSED tests/test_assumptions.py::test_assumed_noop /usr/local/lib/python3.9/site-packages/_pytest/main.py:325: PluggyTeardownRaisedWarning: A plugin raised an exception during an old-style hookwrapper teardown.
Plugin: _cov, Hook: pytest_runtestloop
DataError: Can't combine arc data with line data
For more information see https://pluggy.readthedocs.io/en/stable/api_reference.html#pluggy.PluggyTeardownRaisedWarning
  config.hook.pytest_runtestloop(session=session)

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/_pytest/main.py", line 271, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/_pytest/main.py", line 325, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pluggy/_hooks.py", line 501, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pluggy/_manager.py", line 119, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 155, in _multicall
INTERNALERROR>     teardown[0].send(outcome)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pytest_cov/plugin.py", line 298, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pytest_cov/engine.py", line 44, in ensure_topdir_wrapper
INTERNALERROR>     return meth(self, *args, **kwargs)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pytest_cov/engine.py", line 347, in finish
INTERNALERROR>     self.cov.stop()
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/coverage/control.py", line 832, in combine
INTERNALERROR>     combine_parallel_data(
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/coverage/data.py", line 176, in combine_parallel_data
INTERNALERROR>     data.update(new_data, aliases=aliases)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/coverage/sqldata.py", line 658, in update
INTERNALERROR>     raise DataError("Can't combine arc data with line data")
INTERNALERROR> coverage.exceptions.DataError: Can't combine arc data with line data

========================================================================== 7 failed, 22 passed, 1 error in 36.70s ==========================================================================
*** Error code 3

Version: 4.1.3
Python-3.9
FreeBSD 14.0

@Erotemic
Copy link
Member

This is probably due to customizations in run_tests.py. The logic there could likely be simpler. Will try to look at this later in the week.

@Erotemic
Copy link
Member

Erotemic commented Apr 30, 2024

I got setup with a FreeBSD virtual machine, and I'm running into a different issue about: No module found named _sqlite3.

After I logged into a fresh install I can the following commands:

pkg update
pkg install git
pkg install py39-pip
git clone https://github.com/pyutils/line_profiler.git
cd line_profiler
pip install -e .[tests-strict]
pytest

Do you have any recommendations for how to setup a machine and run tests so I can reproduce this issue?

@yurivict
Copy link
Author

yurivict commented May 1, 2024

I have these instructions on file:

Is there an easy way to test under FreeBSD?

  1. Install the FreeBSD VM image into VirtualBox.
  2. Boot FreeBSD
  3. Install git: pkg install git
  4. Check out the ports tree: git clone https://git.FreeBSD.org/ports.git /usr/ports
  5. cd /usr/ports/devel/py-line-profiler
  6. Install dependencies: pkg install -A `make missing`
  7. Build: make
  8. Run tests: make test

You can begin from step 4 above.

The sqlite module is usually bundled with Python on Linux, but not on FreeBSD.
The ports tree should take care of this.

@Erotemic
Copy link
Member

Erotemic commented May 4, 2024

Working through this slowly. One recommendation I'll make is to update the instructions for git clone to: git clone --depth 1 https://git.FreeBSD.org/ports.git /usr/ports to reduce the amount of data needed. It speeds up the process by a lot.

I'm also noticing that several of the steps have interactive components, e.g. I get prompts like:

image

Is there an adjustment to the above commands to execute them non-interactively, so I can script this?

As an aside: I'm finding it valuable to perform debugging by forwarding a local port to the VM and then ssh-ing into the VM to execute these tasks rather than work with the virtualbox frontend.

@Erotemic
Copy link
Member

Erotemic commented May 4, 2024

The make test just finished, and the errors are not reproducing:

===>   Returning to build of py39-line-profiler-4.1.3                                                                                                                               
===>   py39-line-profiler-4.1.3 depends on package: py39-pytest>=7,1 - found
===>   py39-line-profiler-4.1.3 depends on file: /usr/local/bin/python3.9 - found
cd /usr/ports/devel/py-line-profiler/work-py39/line_profiler-4.1.3 && /usr/bin/env -i HOME=/usr/ports/devel/py-line-profiler/work-py39  PWD="${PWD}"  __MAKE_CONF=/nonexistent OSVERSION=1400097 PATH=/usr/ports/devel/py-line-profiler/work-py39/.bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin TERM=tmux-256color XDG_DATA_HOME=/usr/ports/devel/py-line-profiler/work-py39  XDG_CONFIG_HOME=/usr/ports/devel/py-line-profiler/work-py39  XDG_CACHE_HOME=/usr/ports/devel/py-line-profiler/work-py39/.cache  HOME=/usr/ports/devel/py-line-profiler/work-py39 PATH=/usr/ports/devel/py-line-profiler/work-py39/.bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin PKG_CONFIG_LIBDIR=/usr/ports/devel/py-line-profiler/work-py39/.pkgconfig:/usr/local/libdata/pkgconfig:/usr/local/share/pkgconfig:/usr/libdata/pkgconfig MK_DEBUG_FILES=no MK_KERNEL_SYMBOLS=no SHELL=/bin/sh NO_LINT=YES LDSHARED="cc -shared" PYTHONDONTWRITEBYTECODE= PYTHONOPTIMIZE= PREFIX=/usr/local  LOCALBASE=/usr/local  CC="cc" CFLAGS="-O2 -pipe  -fstack-protector-strong -fno-strict-aliasing "  CPP="cpp" CPPFLAGS=""  LDFLAGS=" -fstack-protector-strong " LIBS=""  CXX="c++" CXXFLAGS="-O2 -pipe -fstack-protector-strong -fno-strict-aliasing  " BSD_INSTALL_PROGRAM="install  -s -m 555"  BSD_INSTALL_LIB="install  -s -m 0644"  BSD_INSTALL_SCRIPT="install  -m 555"  BSD_INSTALL_DATA="install  -m 0644"  BSD_INSTALL_MAN="install  -m 444" PYTHONPATH=/usr/ports/devel/py-line-profiler/work-py39/stage/usr/local/lib/python3.9/site-packages /usr/local/bin/python3.9 -m pytest -k '' -rs -v -o addopts= 
=============================================================================== test session starts ================================================================================
platform freebsd14 -- Python 3.9.18, pytest-8.1.1, pluggy-1.4.0 -- /usr/local/bin/python3.9
cachedir: .pytest_cache
rootdir: /usr/ports/devel/py-line-profiler/work-py39/line_profiler-4.1.3
configfile: pyproject.toml
plugins: xdoctest-1.1.3, cov-3.0.0
collected 34 items                                                                                                                                                                 

tests/test_assumptions.py::test_assumed_noop PASSED                                                                                                                          [  2%]
tests/test_autoprofile.py::test_single_function_autoprofile PASSED                                                                                                           [  5%]
tests/test_autoprofile.py::test_multi_function_autoprofile PASSED                                                                                                            [  8%]
tests/test_autoprofile.py::test_duplicate_function_autoprofile PASSED                                                                                                        [ 11%]
tests/test_autoprofile.py::test_autoprofile_script_with_module PASSED                                                                                                        [ 14%]
tests/test_autoprofile.py::test_autoprofile_module PASSED                                                                                                                    [ 17%]
tests/test_autoprofile.py::test_autoprofile_module_list PASSED                                                                                                               [ 20%]
tests/test_autoprofile.py::test_autoprofile_module_with_prof_imports PASSED                                                                                                  [ 23%]
tests/test_autoprofile.py::test_autoprofile_script_with_prof_imports PASSED                                                                                                  [ 26%]
tests/test_cli.py::test_cli PASSED                                                                                                                                           [ 29%]
tests/test_cli.py::test_version_agreement PASSED                                                                                                                             [ 32%]
tests/test_complex_case.py::test_complex_example_python_none PASSED                                                                                                          [ 35%]
tests/test_complex_case.py::test_varied_complex_invocations PASSED                                                                                                           [ 38%]
tests/test_duplicate_functions.py::test_duplicate_function PASSED                                                                                                            [ 41%]
tests/test_explicit_profile.py::test_simple_explicit_nonglobal_usage PASSED                                                                                                  [ 44%]
tests/test_explicit_profile.py::test_explicit_profile_with_nothing PASSED                                                                                                    [ 47%]
tests/test_explicit_profile.py::test_explicit_profile_with_environ_on PASSED                                                                                                 [ 50%]
tests/test_explicit_profile.py::test_explicit_profile_with_environ_off PASSED                                                                                                [ 52%]
tests/test_explicit_profile.py::test_explicit_profile_with_cmdline PASSED                                                                                                    [ 55%]
tests/test_explicit_profile.py::test_explicit_profile_with_kernprof PASSED                                                                                                   [ 58%]
tests/test_explicit_profile.py::test_explicit_profile_with_in_code_enable PASSED                                                                                             [ 61%]
tests/test_explicit_profile.py::test_explicit_profile_with_duplicate_functions PASSED                                                                                        [ 64%]
tests/test_import.py::test_import PASSED                                                                                                                                     [ 67%]
tests/test_import.py::test_version PASSED                                                                                                                                    [ 70%]
tests/test_ipython.py::TestIPython::test_init PASSED                                                                                                                         [ 73%]
tests/test_kernprof.py::TestKernprof::test_enable_disable PASSED                                                                                                             [ 76%]
tests/test_kernprof.py::TestKernprof::test_function_decorator PASSED                                                                                                         [ 79%]
tests/test_kernprof.py::TestKernprof::test_gen_decorator PASSED                                                                                                              [ 82%]
tests/test_line_profiler.py::test_init PASSED                                                                                                                                [ 85%]
tests/test_line_profiler.py::test_enable_disable PASSED                                                                                                                      [ 88%]
tests/test_line_profiler.py::test_function_decorator PASSED                                                                                                                  [ 91%]
tests/test_line_profiler.py::test_gen_decorator PASSED                                                                                                                       [ 94%]
tests/test_line_profiler.py::test_classmethod_decorator PASSED                                                                                                               [ 97%]
tests/test_line_profiler.py::test_show_func_column_formatting PASSED                                                                                                         [100%]

================================================================================ 34 passed in 3.25s ================================================================================

In your error report, I can't see why tests are failing. Getting stdout or pytest reasons for why the test fail would be helpful.

Also, I'm not sure why coverage is running if the tests just invoke pytest. The custom run_tests.py doesn't seem to be used at all (and that's probably a good thing). But if coverage is somehow enabled, could there be older incompatible coverage files that it is trying to append to on your build system? Even if that is true, I don't think that would explain the failing tests.

A difference I see between your run and my run is that the pytest plugins are different. You have:

platform freebsd14 -- Python 3.9.18, pytest-7.4.4, pluggy-1.4.0 -- /usr/local/bin/python3.9
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase(PosixPath('/usr/ports/devel/py-line-profiler/work-py39/line_profiler-4.1.3/.hypothesis/examples'))
rootdir: /usr/ports/devel/py-line-profiler/work-py39/line_profiler-4.1.3
configfile: pyproject.toml
plugins: anyio-4.3.0, checkdocs-2.12.0, xdoctest-1.1.3, typeguard-4.2.1, hypothesis-6.98.18, enabler-3.1.1, cov-4.1.0, xdist-3.5.0
4 workers [29 items]   error                                                                                                                                                               
scheduling tests via LoadScheduling

Whereas mine is:

platform freebsd14 -- Python 3.9.18, pytest-8.1.1, pluggy-1.4.0 -- /usr/local/bin/python3.9                                                                
cachedir: .pytest_cache                                                                                                                                                             
rootdir: /usr/ports/devel/py-line-profiler/work-py39/line_profiler-4.1.3                                                                                                            
configfile: pyproject.toml                                                                                                                                                          
plugins: xdoctest-1.1.3, cov-3.0.0                                                                                                                                                  
collected 34 items                                                                                                                                                                                               

My pytest version is also different. Is there a modification to the above commands that could more faithfully reproduce your build environment?

@Erotemic
Copy link
Member

Erotemic commented May 4, 2024

I attempted to reproduce the versions via:

/usr/local/bin/python3.9 -m pip uninstall pytest
/usr/local/bin/python3.9 -m pip uninstall pytest
pip install "pytest==7.4.4"
pip install anyio
pip install "pytest-checkdocs==2.12.0"
pip install "pytest==7.4.4"
pip install "typeguard==4.2.1"
pip install "hypothesis==6.98.18"
pip install "pytest-cov==4.1.0"
pip install "pytest-enabler==3.1.1" "pytest-xdist==3.5.0"
/usr/local/bin/python3.9 -m pytest -k '' -rs -v -o addopts=

which resulted in

root@foobar:/usr/ports/devel/py-line-profiler/work-py39/line_profiler-4.1.3 # /usr/local/bin/python3.9 -m pytest -k '' -rs -v -o addopts=
=============================================================================== test session starts ================================================================================
platform freebsd14 -- Python 3.9.18, pytest-7.4.4, pluggy-1.4.0 -- /usr/local/bin/python3.9
                                                                                                                                                                                    
cachedir: .pytest_cache                                                                   
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase(PosixPath('/usr/ports/devel/py-line-profiler/work-py39/line_profiler-4.1.3/.hypothesis/examples'))           
rootdir: /usr/ports/devel/py-line-profiler/work-py39/line_profiler-4.1.3
configfile: pyproject.toml                                                                                                                                                          
plugins: xdoctest-1.1.3, cov-4.1.0, anyio-4.3.0, typeguard-4.2.1, hypothesis-6.98.18, enabler-3.1.1, xdist-3.5.0, checkdocs-2.12.0
8 workers [35 items]                                                                                                                                                                
scheduling tests via LoadScheduling                                                       


... omitted for clarity ...

INTERNALERROR> coverage.exceptions.DataError: Can't combine arc data with line data

================================================================================ 35 passed in 5.63s ================================================================================

So I was able to reproduce the coverage issue, but all 34 tests are still passing just fine for me. I'd be interested in seeing what the result of this command is on your system:

/usr/local/bin/python3.9 -m pytest -p no:xdist -p no:cov -k '' -s -v -o addopts=

The important parts being change -rs to -s to show stdout, add -p no:xdist to disable the xdist plugin, otherwise stdout wont show, and add -p no:cov to disable coverage as it is not necessary here and it works around the arc/line-data issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants