Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Make conda builds for all platforms and Python versions #17

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

kohr-h
Copy link
Member

@kohr-h kohr-h commented Jun 12, 2016

I added the meta.yaml file and a build.sh build script for Linux and Mac. For windows builds, we need a bld.bat file like this one, just with proper build instructions.

What's currently known to work:

Linux:

  • build
  • install
  • import
  • tests succeed

EDIT: I took the package built on my office computer and installed it on my laptop, where CUDA is located in a different directory. Works nicely.

Mac:

  • build
  • install
  • import
  • tests succeed

Windows:

  • build
  • install
  • import
  • tests succeed

Instruction on how to build a conda package are here.

Especially environment variables that can be used during build are listed here.

EDIT: Forgot to mention that the build expects the environment variables CUDA_ROOT and CUDA_COMPUTE to be defined. In Linux, you would run

CUDA_ROOT=/opt/cuda CUDA_COMPUTE=30 conda build .

in the conda directory.

- pytest

about:
home: https://github.com/odlgroup/odl
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not updated obviously

- cudatoolkit
- boost
# Make sure to run "conda config --append channels odlgroup"
- odl >=0.3.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0.4.0

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we want to keep the version perfectly in sync, so probably a version after the split is sufficient. It will anyway pick the most recent one.

@adler-j
Copy link
Member

adler-j commented Aug 22, 2016

Very good initiative, I'll try to get windows settled.

BOOST_LIBRARY=$PREFIX/lib/libboost_python.so

ODLCUDA_BUILDDIR=$SRC_DIR/build-py$PYTHON_VERSION
ODLCUDA_NUMPY_BUILDOPTS="-DPYTHON_NUMPY_INCLUDE_DIR=$PREFIX/lib/python$PYTHON_VERSION/site-packages/numpy/core/include"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should use

$ python -c "import numpy; print(numpy.get_include())"

@kohr-h
Copy link
Member Author

kohr-h commented Aug 22, 2016

Now everything works as expected with conda's version of boost and CUDA-toolkit. That should also make it easier to get the Windows build working since the paths are more predictable. I've uploaded the last build to Anaconda cloud, users can smoothly install from there.

@adler-j
Copy link
Member

adler-j commented Sep 22, 2016

I'll see if i can get this working on windows as well

@kohr-h
Copy link
Member Author

kohr-h commented Sep 22, 2016

I'll see if i can get this working on windows as well

Would be great! I can summarize some of my experiences:

  • Despite of what I wrote above, the CUDA toolkit from conda didn't work as expected, but boost is fine. I'll keep trying to make also work, though. Alternatively, one could use the conda CUDA for the Windows build and a distribution CUDA for Unix builds.
  • Something fails during the make pyinstall step in the build when using Python 2.7. It tries to download ODL during that step although it's installed as build dependency.
  • Don't use conda build from within a conda environment, it can put you into an infinite loop (with conda-build >= 2.0.0. Use the root environment and give --python <version> as option to build for a certain Python version.
  • Set CUDA_COMPUTE to 50: lower values fail the package to be usable for some reason, higher values won't work with machines without that compute capability.

@adler-j
Copy link
Member

adler-j commented Sep 29, 2016

Proposal:

Go ahead with Linux, I'll catch up with windows "sometime".

@kohr-h kohr-h changed the title [WIP] Make conda builds of odlpp for all platforms [WIP] Make conda builds for all platforms and Python versions Oct 21, 2016
@kohr-h
Copy link
Member Author

kohr-h commented Oct 21, 2016

When installing on a new machine using conda, I got a spurious "invalid device function" error. I definitely have a recent enough GPU, so too low compute capability is not the problem. For some reason, I experienced the same when building with CUDA_COMPUTE=30 and installing on a machine with "better" GPU. Any idea what happens @adler-j? I hope it's no problem that I have CUDA 8 here, that would make the build complexity explode.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants