Installation issues: 'p2ptrans --help' terminated by signal SIGSEGV (Address boundary error)

[mkhorton]

'p2ptrans --help' terminated by signal SIGSEGV (Address boundary error)

Tried both CXXFLAGS="-std=c++11" pip install git+https://github.com/ftherrien/p2ptrans and pip install git+https://github.com/ftherrien/p2ptrans

pip freeze:

appnope==0.1.0
attrs==19.3.0
backcall==0.1.0
bleach==3.1.1
certifi==2019.11.28
cycler==0.10.0
Cython==0.29.15
decorator==4.4.1
defusedxml==0.6.0
entrypoints==0.3
f90nml==1.1.2
glob2==0.7
importlib-metadata==1.5.0
ipykernel==5.1.4
ipython==7.12.0
ipython-genutils==0.2.0
jedi==0.16.0
Jinja2==2.11.1
jsonschema==3.2.0
jupyter-client==6.0.0
jupyter-core==4.6.3
kiwisolver==1.1.0
Mako==1.1.1
MarkupSafe==1.1.1
matplotlib==3.1.3
mistune==0.8.4
more-itertools==8.2.0
nbconvert==5.6.1
nbformat==5.0.4
numpy==1.18.1
-e git+https://github.com/ftherrien/p2ptrans.git@93e32a8485e66bf102563ed669091bfbf76da732#egg=p2ptrans
packaging==20.1
pandocfilters==1.4.2
parse==1.14.0
parse-type==0.5.2
parso==0.6.1
pexpect==4.8.0
pickleshare==0.7.5
pluggy==0.13.1
prompt-toolkit==3.0.3
ptyprocess==0.6.0
py==1.8.1
Pygments==2.5.2
pylada==1.0
pyparsing==2.4.6
pyrsistent==0.15.7
pytest==5.3.5
pytest-bdd==3.2.1
python-dateutil==2.8.1
pyzmq==19.0.0
quantities==0.12.4
scipy==1.4.1
six==1.14.0
spglib==1.14.1.post0
testpath==0.4.4
tornado==6.0.3
traitlets==4.3.3
wcwidth==0.1.8
webencodings==0.5.1
zipp==3.0.0

and Python 3.7.6 on macOS 10.15.2

[mkhorton]

Btw, I'm not sure what the "on certain systems" issue is, but if it's related to libstdc++ no longer being included on Mac, there's a solution to this:

https://github.com/pandas-dev/pandas/blob/8e4424fbcfc98d23085ddbcf8765946bc85da035/setup.py#L477

[ftherrien]

@mkhorton I haven't gotten a chance to test the latest updates on Mac yet. Let me work on it and I will get back to you very soon

[mkhorton]

No worries, happy to help too if I can. Github Actions is a nice way to do cross-platform testing, you can write a single test which is just installing p2ptrans and run it in a matrix of OSs [Windows, macOS, Ubuntu, etc.] and flag if there's an error.

Forgot to add that I was using gfortran 9.2.0 for compilation.

[ftherrien]

@mkhorton I was not able to reproduce your problem. I tested the installation and p2ptrans --help on Sierra and on Mojave without any issue.

Mojave 10.14.5
gfortran 9.2.0
make 3.16.3
python 3.8.2

What steps did you take exactly? Are you installing in a virtual environment, a conda environment, using --user or system wide? It seems like you were installing a particular commit, did you use the -e option? is the local bin folder in your PATH?

[mkhorton]

Gfortran via homebrew, in a fresh conda environment as listed above. Installed dependencies (starting numpy), initially tried a “python setup.py develop” but also then tried installing directly from the git repo instead (as listed in your README).

…

On Tue, Mar 3, 2020 at 08:48, Félix Therrien ***@***.***> wrote: @mkhorton <https://github.com/mkhorton> I was not able to reproduce your problem. I tested the installation and p2ptrans --help on Sierra and on Mojave without any issue. Mojave 10.14.5 gfortran 9.2.0 make 3.16.3 python 3.8.2 What steps did you take exactly? Are you installing in a virtual environment, a conda environment, using --user or system wide? It seems like you were installing a particular commit, did you use the -e option? is the local bin folder in your PATH? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#8?email_source=notifications&email_token=AAWWWRBB5IOGFR7YR57Y2D3RFUYGDA5CNFSM4K67PBBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENUH6NI#issuecomment-594050869>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAWWWRENRINRBEFYGNDW2WLRFUYGDANCNFSM4K67PBBA> .

[ftherrien]

@mkhorton If I understand correctly, the installation is successful but the program fails when you try to run it. Is that correct?

Note:
python setup.py develop will not use PEP517 and it should fail (in a fresh conda envrionment with no site-packages) because it can't import numpy. Is that what happened at first? Try using pip install . instead making sure pip is upgraded to at least version 19.0, then you won't need to install numpy first.

After the installation completes. What happens if you try to import p2ptrans in python? e.g.python -c "import p2ptrans"

If that works fine, what is the first line in the file p2ptrans located in your conda environment's bin?
e.g. head -1 $(which p2ptrans)

[mkhorton]

Yes, that's correct, I get a segfault and Python itself crashes.

Is that what happened at first?

Yeah, I made sure to install all requirements first.

What happens if you try to import p2ptrans in python?

The same error.

python -c "import p2ptrans"' terminated by signal SIGSEGV (Address boundary error)

which p2ptrans gives /usr/local/miniconda3/envs/p2ptrans/bin/p2ptrans

[ftherrien]

@mkhorton I think I was able to reproduce the error! I get it only when using a conda environment on MacOS.

It seems like pip failed to build the wheel but was still able to finish installing anyway:

fatal error: 'numpy/arrayobject.h' file not found
  #include "numpy/arrayobject.h"

Do you see that somewhere in your install output? Anyway, if that is the correct error then this is the same as this issue: hmmlearn/hmmlearn#43 and this:

CFLAGS="-I $(python -c "import numpy; print(numpy.get_include())") $CFLAGS" pip install git+https://github.com/ftherrien/p2ptrans

should solve it.

[mkhorton]

Frustratingly, still seeing the error, despite claiming the wheel built successfully:

> CFLAGS="-I /usr/local/miniconda3/envs/p2ptrans/lib/python3.7/site-packages/numpy/core/include $CFLAGS" pip install git+https://github.com/ftherrien/p2ptrans
... 
Building wheels for collected packages: p2ptrans
  Building wheel for p2ptrans (PEP 517) ... done
  Created wheel for p2ptrans: filename=p2ptrans-0.0-cp37-cp37m-macosx_10_9_x86_64.whl size=178484 sha256=f635a896230c61321d7cc83bd26ca5d9fc2653f29dd060a925985906bfab0d65
  Stored in directory: /private/var/folders/m5/nj63sfvd3n91h2783hqp37_40000gn/T/pip-ephem-wheel-cache-bh9axhpg/wheels/ae/19/b0/e27c11384d777b9817c52bc01551e8a4a045242d8e5011c325
Successfully built p2ptrans
Installing collected packages: p2ptrans
Successfully installed p2ptrans-0.0

> p2ptrans --help
'p2ptrans --help' terminated by signal SIGSEGV (Address boundary error)

However, your intuition about this being a conda issue seems to be correct -- I installed via a clean virtualenv and it seems to work fine :-) Why conda is presenting a problem I'm not sure.

[ftherrien]

Ah! Yes you are right! In my excitement I did not try the actual faulty p2ptrans --help. The problem is with the pylada installation not with p2ptrans. There is an open issue about this here: pylada/pylada-light#42. The problem occurs specifically with conda on MacOS. There does not seem to be a solution yet...

[mkhorton]

Ah ok, thanks for the info! What aspects of pylada does p2ptrans depend upon?

[ftherrien]

It depends on multiple functions and classes of pylada...

[ftherrien]

Ok here is a solution: (in a fresh conda env)

pip install git+https://github.com/pylada/pylada-light.git@scikit
pip install git+https://github.com/ftherrien/p2ptrans

[mkhorton]

I can confirm that works!! Also just noting the setup.py has pylada not pylada-light, I'm not sure if this makes any practical difference. I wonder if pre-built wheels on PyPI would avoid this issue?

[ftherrien]

pylada-light is just the name of the repo, the package name is pylada. Of course putting both pylada and p2ptrans on PyPl would probably be better! See #9

[mdavezac]

I've just merged a branch in pylada that builds pylada with scikit-build. It also tests macos + conda via a github action. So now it should be sufficient to do pip install git+https://github.com/ftherrien/p2ptrans alone.

[ftherrien]

@mdavezac Thank you so much!