Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix building torch extension with glog #1791

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

petronny
Copy link

@petronny petronny commented Mar 31, 2024

To fix the following error when building python-apex-git on Arch Linux:

running build_ext
/usr/lib/python3.11/site-packages/torch/utils/cpp_extension.py:500: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
  warnings.warn(msg.format('we could not find ninja.'))
/usr/lib/python3.11/site-packages/torch/utils/cpp_extension.py:425: UserWarning: There are no /opt/cuda/bin/g++ version bounds defined for CUDA version 12.4
  warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
building 'apex_C' extension
creating build/temp.linux-x86_64-cpython-311
creating build/temp.linux-x86_64-cpython-311/csrc
/opt/cuda/bin/gcc -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -flto=auto -fPIC -I/usr/lib/python3.11/site-packages/torch/include -I/usr/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/usr/lib/python3.11/site-packages/torch/include/TH -I/usr/lib/python3.11/site-packages/torch/include/THC -I/usr/include/python3.11 -c csrc/flatten_unflatten.cpp -o build/temp.linux-x86_64-cpython-311/csrc/flatten_unflatten.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1018\" -DTORCH_EXTENSION_NAME=apex_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++17
In file included from /usr/include/c10/util/logging_is_google_glog.h:50,
                 from /usr/include/c10/util/Logging.h:26,
                 from /usr/include/ATen/core/ivalue_inl.h:25,
                 from /usr/include/ATen/core/ivalue.h:1551,
                 from /usr/include/ATen/core/List_inl.h:4,
                 from /usr/include/ATen/core/List.h:490,
                 from /usr/include/ATen/core/IListRef_inl.h:3,
                 from /usr/include/ATen/core/IListRef.h:631,
                 from /usr/include/ATen/WrapDimUtils.h:3,
                 from /usr/include/ATen/TensorNames.h:3,
                 from /usr/include/ATen/NamedTensorUtils.h:3,
                 from /usr/include/torch/csrc/autograd/variable.h:11,
                 from /usr/include/torch/csrc/autograd/autograd.h:3,
                 from /usr/lib/python3.11/site-packages/torch/include/torch/csrc/api/include/torch/autograd.h:3,
                 from /usr/lib/python3.11/site-packages/torch/include/torch/csrc/api/include/torch/all.h:7,
                 from /usr/include/torch/extension.h:5,
                 from csrc/flatten_unflatten.cpp:1:
/usr/include/glog/logging.h:60:4: error: #error <glog/logging.h> was not included correctly. See the documention for how to consume the library.
   60 | #  error <glog/logging.h> was not included correctly. See the documention for how to consume the library.
      |    ^~~~~

Full build log: https://github.com/arch4edu/cactus/actions/runs/8490889706/job/23262378384

Ping @hubutui

@hubutui
Copy link

hubutui commented Mar 31, 2024

If torch upstream could add a default macro GLOG_USE_GLOG_EXPORT, then all downstream who need to compile torch extension could work out-of-box.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants