Skip to content
This repository has been archived by the owner on Oct 16, 2023. It is now read-only.

Does not support Cuda 10.2 ? #193

Open
0-1CxH opened this issue Feb 22, 2023 · 1 comment
Open

Does not support Cuda 10.2 ? #193

0-1CxH opened this issue Feb 22, 2023 · 1 comment

Comments

@0-1CxH
Copy link

0-1CxH commented Feb 22, 2023

When installing this with pip install . , the following error is encountered:

fatal error: cub/cub.cuh: No such file or directory
       #include <cub/cub.cuh>
                ^~~~~~~~~~~~~
      compilation terminated.

The complete log is:

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Processing /root/gpt_exp/opt_colossal/EnergonAI-main
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: energonai
  Building wheel for energonai (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [106 lines of output]


      torch.__version__  = 1.10.1+cu102



      Compiling cuda extensions with
      nvcc: NVIDIA (R) Cuda compiler driver
      Copyright (c) 2005-2019 NVIDIA Corporation
      Built on Wed_Oct_23_19:24:38_PDT_2019
      Cuda compilation tools, release 10.2, V10.2.89
      from /usr/local/cuda/bin

      running bdist_wheel
      running build
      running build_py
      running build_ext
      building 'energonai_scale_mask' extension
      /home/kg/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py:298: UserWarning:

                                     !! WARNING !!

      !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
      Your compiler (c++) is not compatible with the compiler Pytorch was
      built with for this platform, which is g++ on linux. Please
      use g++ to to compile your extension. Alternatively, you may
      compile PyTorch from source using c++, and then you can also use
      c++ to compile your extension.

      See https://github.com/pytorch/pytorch/blob/master/CONTRIBUTING.md for help
      with compiling PyTorch from source.
      !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

                                    !! WARNING !!

        platform=sys.platform))
      Emitting ninja build file /root/gpt_exp/opt_colossal/EnergonAI-main/build/temp.linux-x86_64-3.7/build.ninja...
      Compiling objects...
      Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
      [1/1] /usr/local/cuda/bin/nvcc  -I/root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc -I/home/kg/anaconda3/lib/python3.7/site-packages/torch/include -I/home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/kg/anaconda3/lib/python3.7/site-packages/torch/include/TH -I/home/kg/anaconda3/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/kg/anaconda3/include/python3.7m -c -c /root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc/scale_mask_softmax_kernel.cu -o /root/gpt_exp/opt_colossal/EnergonAI-main/build/temp.linux-x86_64-3.7/energonai/kernel/cuda_native/csrc/scale_mask_softmax_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 --use_fast_math -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DUSE_C10D_NCCL -std=c++14 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_HALF2_OPERATORS__ -DTHRUST_IGNORE_CUB_VERSION_CHECK -gencode arch=compute_70,code=sm_70 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=energonai_scale_mask -D_GLIBCXX_USE_CXX11_ABI=0
      FAILED: /root/gpt_exp/opt_colossal/EnergonAI-main/build/temp.linux-x86_64-3.7/energonai/kernel/cuda_native/csrc/scale_mask_softmax_kernel.o
      /usr/local/cuda/bin/nvcc  -I/root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc -I/home/kg/anaconda3/lib/python3.7/site-packages/torch/include -I/home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/kg/anaconda3/lib/python3.7/site-packages/torch/include/TH -I/home/kg/anaconda3/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/kg/anaconda3/include/python3.7m -c -c /root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc/scale_mask_softmax_kernel.cu -o /root/gpt_exp/opt_colossal/EnergonAI-main/build/temp.linux-x86_64-3.7/energonai/kernel/cuda_native/csrc/scale_mask_softmax_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 --use_fast_math -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DUSE_C10D_NCCL -std=c++14 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_HALF2_OPERATORS__ -DTHRUST_IGNORE_CUB_VERSION_CHECK -gencode arch=compute_70,code=sm_70 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=energonai_scale_mask -D_GLIBCXX_USE_CXX11_ABI=0
      /root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc/scale_mask_softmax_kernel.cu:5:10: fatal error: cub/cub.cuh: No such file or directory
       #include <cub/cub.cuh>
                ^~~~~~~~~~~~~
      compilation terminated.
      ninja: build stopped: subcommand failed.
      Traceback (most recent call last):
        File "/home/kg/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1723, in _run_ninja_build
          env=env)
        File "/home/kg/anaconda3/lib/python3.7/subprocess.py", line 487, in run
          output=stdout, stderr=stderr)
      subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

      The above exception was the direct cause of the following exception:

      Traceback (most recent call last):
        File "<string>", line 36, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/root/gpt_exp/opt_colossal/EnergonAI-main/setup.py", line 187, in <module>
          'console_scripts': ['energonai=energonai.cli:typer_click_object', ],
        File "/home/kg/anaconda3/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
          return distutils.core.setup(**attrs)
        File "/home/kg/anaconda3/lib/python3.7/distutils/core.py", line 148, in setup
          dist.run_commands()
        File "/home/kg/anaconda3/lib/python3.7/distutils/dist.py", line 966, in run_commands
          self.run_command(cmd)
        File "/home/kg/anaconda3/lib/python3.7/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/home/kg/anaconda3/lib/python3.7/site-packages/wheel/bdist_wheel.py", line 299, in run
          self.run_command('build')
        File "/home/kg/anaconda3/lib/python3.7/distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/home/kg/anaconda3/lib/python3.7/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/home/kg/anaconda3/lib/python3.7/distutils/command/build.py", line 135, in run
          self.run_command(cmd_name)
        File "/home/kg/anaconda3/lib/python3.7/distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/home/kg/anaconda3/lib/python3.7/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/home/kg/anaconda3/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 79, in run
          _build_ext.run(self)
        File "/home/kg/anaconda3/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
          _build_ext.build_ext.run(self)
        File "/home/kg/anaconda3/lib/python3.7/distutils/command/build_ext.py", line 340, in run
          self.build_extensions()
        File "/home/kg/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 735, in build_extensions
          build_ext.build_extensions(self)
        File "/home/kg/anaconda3/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/home/kg/anaconda3/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
          self._build_extensions_serial()
        File "/home/kg/anaconda3/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
          self.build_extension(ext)
        File "/home/kg/anaconda3/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
          _build_ext.build_extension(self, ext)
        File "/home/kg/anaconda3/lib/python3.7/distutils/command/build_ext.py", line 534, in build_extension
          depends=ext.depends)
        File "/home/kg/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 565, in unix_wrap_ninja_compile
          with_cuda=with_cuda)
        File "/home/kg/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1404, in _write_ninja_file_and_compile_objects
          error_prefix='Error compiling objects for extension')
        File "/home/kg/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1733, in _run_ninja_build
          raise RuntimeError(message) from e
      RuntimeError: Error compiling objects for extension
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for energonai
  Running setup.py clean for energonai
Failed to build energonai
Installing collected packages: energonai
  Running setup.py install for energonai ... error
  error: subprocess-exited-with-error

  × Running setup.py install for energonai did not run successfully.
  │ exit code: 1
  ╰─> [272 lines of output]


      torch.__version__  = 1.10.1+cu102



      Compiling cuda extensions with
      nvcc: NVIDIA (R) Cuda compiler driver
      Copyright (c) 2005-2019 NVIDIA Corporation
      Built on Wed_Oct_23_19:24:38_PDT_2019
      Cuda compilation tools, release 10.2, V10.2.89
      from /usr/local/cuda/bin

      running install
      /home/kg/anaconda3/lib/python3.7/site-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        setuptools.SetuptoolsDeprecationWarning,
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-3.7
      creating build/lib.linux-x86_64-3.7/energonai
      copying energonai/worker.py -> build/lib.linux-x86_64-3.7/energonai
      copying energonai/task.py -> build/lib.linux-x86_64-3.7/energonai
      copying energonai/__init__.py -> build/lib.linux-x86_64-3.7/energonai
      copying energonai/engine.py -> build/lib.linux-x86_64-3.7/energonai
      copying energonai/batch_mgr.py -> build/lib.linux-x86_64-3.7/energonai
      copying energonai/pipe.py -> build/lib.linux-x86_64-3.7/energonai
      creating build/lib.linux-x86_64-3.7/energonai/utils
      copying energonai/utils/files.py -> build/lib.linux-x86_64-3.7/energonai/utils
      copying energonai/utils/checkpointing_hf_gpt2.py -> build/lib.linux-x86_64-3.7/energonai/utils
      copying energonai/utils/timer.py -> build/lib.linux-x86_64-3.7/energonai/utils
      copying energonai/utils/__init__.py -> build/lib.linux-x86_64-3.7/energonai/utils
      copying energonai/utils/common.py -> build/lib.linux-x86_64-3.7/energonai/utils
      copying energonai/utils/checkpointing_opt.py -> build/lib.linux-x86_64-3.7/energonai/utils
      copying energonai/utils/checkpointing.py -> build/lib.linux-x86_64-3.7/energonai/utils
      creating build/lib.linux-x86_64-3.7/energonai/testing
      copying energonai/testing/models.py -> build/lib.linux-x86_64-3.7/energonai/testing
      copying energonai/testing/__init__.py -> build/lib.linux-x86_64-3.7/energonai/testing
      creating build/lib.linux-x86_64-3.7/energonai/model
      copying energonai/model/attention.py -> build/lib.linux-x86_64-3.7/energonai/model
      copying energonai/model/embedding.py -> build/lib.linux-x86_64-3.7/energonai/model
      copying energonai/model/__init__.py -> build/lib.linux-x86_64-3.7/energonai/model
      copying energonai/model/mlp.py -> build/lib.linux-x86_64-3.7/energonai/model
      copying energonai/model/endecoder.py -> build/lib.linux-x86_64-3.7/energonai/model
      copying energonai/model/model_factory.py -> build/lib.linux-x86_64-3.7/energonai/model
      copying energonai/model/downstream.py -> build/lib.linux-x86_64-3.7/energonai/model
      creating build/lib.linux-x86_64-3.7/energonai/communication
      copying energonai/communication/collective.py -> build/lib.linux-x86_64-3.7/energonai/communication
      copying energonai/communication/p2p.py -> build/lib.linux-x86_64-3.7/energonai/communication
      copying energonai/communication/__init__.py -> build/lib.linux-x86_64-3.7/energonai/communication
      copying energonai/communication/utils.py -> build/lib.linux-x86_64-3.7/energonai/communication
      copying energonai/communication/ring.py -> build/lib.linux-x86_64-3.7/energonai/communication
      creating build/lib.linux-x86_64-3.7/energonai/legacy_batch_mgr
      copying energonai/legacy_batch_mgr/dynamic_batch_manager.py -> build/lib.linux-x86_64-3.7/energonai/legacy_batch_mgr
      copying energonai/legacy_batch_mgr/__init__.py -> build/lib.linux-x86_64-3.7/energonai/legacy_batch_mgr
      copying energonai/legacy_batch_mgr/naive_batch_manager.py -> build/lib.linux-x86_64-3.7/energonai/legacy_batch_mgr
      creating build/lib.linux-x86_64-3.7/energonai/pipelinable
      copying energonai/pipelinable/split_method.py -> build/lib.linux-x86_64-3.7/energonai/pipelinable
      copying energonai/pipelinable/__init__.py -> build/lib.linux-x86_64-3.7/energonai/pipelinable
      copying energonai/pipelinable/energon_tracer.py -> build/lib.linux-x86_64-3.7/energonai/pipelinable
      copying energonai/pipelinable/split_policy.py -> build/lib.linux-x86_64-3.7/energonai/pipelinable
      creating build/lib.linux-x86_64-3.7/energonai/kernel
      copying energonai/kernel/__init__.py -> build/lib.linux-x86_64-3.7/energonai/kernel
      creating build/lib.linux-x86_64-3.7/energonai/kernel/cuda_native
      copying energonai/kernel/cuda_native/__init__.py -> build/lib.linux-x86_64-3.7/energonai/kernel/cuda_native
      copying energonai/kernel/cuda_native/scale_mask_softmax.py -> build/lib.linux-x86_64-3.7/energonai/kernel/cuda_native
      copying energonai/kernel/cuda_native/transpose_pad.py -> build/lib.linux-x86_64-3.7/energonai/kernel/cuda_native
      copying energonai/kernel/cuda_native/linear_func.py -> build/lib.linux-x86_64-3.7/energonai/kernel/cuda_native
      copying energonai/kernel/cuda_native/layer_norm.py -> build/lib.linux-x86_64-3.7/energonai/kernel/cuda_native
      copying energonai/kernel/cuda_native/one_layer_norm.py -> build/lib.linux-x86_64-3.7/energonai/kernel/cuda_native
      running build_ext
      building 'energonai_scale_mask' extension
      creating /root/gpt_exp/opt_colossal/EnergonAI-main/build/temp.linux-x86_64-3.7
      creating /root/gpt_exp/opt_colossal/EnergonAI-main/build/temp.linux-x86_64-3.7/energonai
      creating /root/gpt_exp/opt_colossal/EnergonAI-main/build/temp.linux-x86_64-3.7/energonai/kernel
      creating /root/gpt_exp/opt_colossal/EnergonAI-main/build/temp.linux-x86_64-3.7/energonai/kernel/cuda_native
      creating /root/gpt_exp/opt_colossal/EnergonAI-main/build/temp.linux-x86_64-3.7/energonai/kernel/cuda_native/csrc
      /home/kg/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py:298: UserWarning:

                                     !! WARNING !!

      !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
      Your compiler (c++) is not compatible with the compiler Pytorch was
      built with for this platform, which is g++ on linux. Please
      use g++ to to compile your extension. Alternatively, you may
      compile PyTorch from source using c++, and then you can also use
      c++ to compile your extension.

      See https://github.com/pytorch/pytorch/blob/master/CONTRIBUTING.md for help
      with compiling PyTorch from source.
      !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

                                    !! WARNING !!

        platform=sys.platform))
      Emitting ninja build file /root/gpt_exp/opt_colossal/EnergonAI-main/build/temp.linux-x86_64-3.7/build.ninja...
      Compiling objects...
      Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
      [1/2] /usr/local/cuda/bin/nvcc  -I/root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc -I/home/kg/anaconda3/lib/python3.7/site-packages/torch/include -I/home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/kg/anaconda3/lib/python3.7/site-packages/torch/include/TH -I/home/kg/anaconda3/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/kg/anaconda3/include/python3.7m -c -c /root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc/scale_mask_softmax_kernel.cu -o /root/gpt_exp/opt_colossal/EnergonAI-main/build/temp.linux-x86_64-3.7/energonai/kernel/cuda_native/csrc/scale_mask_softmax_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 --use_fast_math -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DUSE_C10D_NCCL -std=c++14 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_HALF2_OPERATORS__ -DTHRUST_IGNORE_CUB_VERSION_CHECK -gencode arch=compute_70,code=sm_70 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=energonai_scale_mask -D_GLIBCXX_USE_CXX11_ABI=0
      FAILED: /root/gpt_exp/opt_colossal/EnergonAI-main/build/temp.linux-x86_64-3.7/energonai/kernel/cuda_native/csrc/scale_mask_softmax_kernel.o
      /usr/local/cuda/bin/nvcc  -I/root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc -I/home/kg/anaconda3/lib/python3.7/site-packages/torch/include -I/home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/kg/anaconda3/lib/python3.7/site-packages/torch/include/TH -I/home/kg/anaconda3/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/kg/anaconda3/include/python3.7m -c -c /root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc/scale_mask_softmax_kernel.cu -o /root/gpt_exp/opt_colossal/EnergonAI-main/build/temp.linux-x86_64-3.7/energonai/kernel/cuda_native/csrc/scale_mask_softmax_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 --use_fast_math -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DUSE_C10D_NCCL -std=c++14 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_HALF2_OPERATORS__ -DTHRUST_IGNORE_CUB_VERSION_CHECK -gencode arch=compute_70,code=sm_70 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=energonai_scale_mask -D_GLIBCXX_USE_CXX11_ABI=0
      /root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc/scale_mask_softmax_kernel.cu:5:10: fatal error: cub/cub.cuh: No such file or directory
       #include <cub/cub.cuh>
                ^~~~~~~~~~~~~
      compilation terminated.
      [2/2] c++ -MMD -MF /root/gpt_exp/opt_colossal/EnergonAI-main/build/temp.linux-x86_64-3.7/energonai/kernel/cuda_native/csrc/scale_mask_softmax_wrapper.o.d -pthread -B /home/kg/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc -I/home/kg/anaconda3/lib/python3.7/site-packages/torch/include -I/home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/kg/anaconda3/lib/python3.7/site-packages/torch/include/TH -I/home/kg/anaconda3/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/kg/anaconda3/include/python3.7m -c -c /root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc/scale_mask_softmax_wrapper.cpp -o /root/gpt_exp/opt_colossal/EnergonAI-main/build/temp.linux-x86_64-3.7/energonai/kernel/cuda_native/csrc/scale_mask_softmax_wrapper.o -O3 -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DUSE_C10D_NCCL -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=energonai_scale_mask -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
      cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
      In file included from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Device.h:5,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Allocator.h:6,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/ATen/ATen.h:7,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/extension.h:4,
                       from /root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc/scale_mask_softmax_wrapper.cpp:2:
      /root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc/scale_mask_softmax_wrapper.cpp: In function 'at::Tensor scale_mask_softmax_wrapper(int, int, int, at::Tensor, at::Tensor)':
      /root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc/scale_mask_softmax_wrapper.cpp:5:21: warning: 'at::DeprecatedTypeProperties& at::Tensor::type() const' is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
         AT_ASSERTM(x.type().is_cuda(), #x " must be a CUDA tensor")
                           ^
      /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:241:39: note: in definition of macro 'C10_EXPAND_MSVC_WORKAROUND'
       #define C10_EXPAND_MSVC_WORKAROUND(x) x
                                             ^
      /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:261:34: note: in expansion of macro 'C10_UNLIKELY'
       #define C10_UNLIKELY_OR_CONST(e) C10_UNLIKELY(e)
                                        ^~~~~~~~~~~~
      /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:313:7: note: in expansion of macro 'C10_UNLIKELY_OR_CONST'
         if (C10_UNLIKELY_OR_CONST(!(cond))) {                                         \
             ^~~~~~~~~~~~~~~~~~~~~
      /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:599:32: note: in expansion of macro 'TORCH_INTERNAL_ASSERT'
           C10_EXPAND_MSVC_WORKAROUND(TORCH_INTERNAL_ASSERT(cond, __VA_ARGS__)); \
                                      ^~~~~~~~~~~~~~~~~~~~~
      /root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc/scale_mask_softmax_wrapper.cpp:5:3: note: in expansion of macro 'AT_ASSERTM'
         AT_ASSERTM(x.type().is_cuda(), #x " must be a CUDA tensor")
         ^~~~~~~~~~
      /root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc/scale_mask_softmax_wrapper.cpp:13:3: note: in expansion of macro 'CHECK_CUDA'
         CHECK_CUDA(x);                                                               \
         ^~~~~~~~~~
      /root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc/scale_mask_softmax_wrapper.cpp:29:3: note: in expansion of macro 'CHECK_FP16_32_INPUT'
         CHECK_FP16_32_INPUT(correlation);
         ^~~~~~~~~~~~~~~~~~~
      In file included from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/ATen/Tensor.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/ATen/Context.h:4,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/ATen/ATen.h:9,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/extension.h:4,
                       from /root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc/scale_mask_softmax_wrapper.cpp:2:
      /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/ATen/core/TensorBody.h:194:30: note: declared here
         DeprecatedTypeProperties & type() const {
                                    ^~~~
      In file included from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Device.h:5,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/c10/core/Allocator.h:6,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/ATen/ATen.h:7,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/extension.h:4,
                       from /root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc/scale_mask_softmax_wrapper.cpp:2:
      /root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc/scale_mask_softmax_wrapper.cpp:5:21: warning: 'at::DeprecatedTypeProperties& at::Tensor::type() const' is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
         AT_ASSERTM(x.type().is_cuda(), #x " must be a CUDA tensor")
                           ^
      /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:241:39: note: in definition of macro 'C10_EXPAND_MSVC_WORKAROUND'
       #define C10_EXPAND_MSVC_WORKAROUND(x) x
                                             ^
      /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:261:34: note: in expansion of macro 'C10_UNLIKELY'
       #define C10_UNLIKELY_OR_CONST(e) C10_UNLIKELY(e)
                                        ^~~~~~~~~~~~
      /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:313:7: note: in expansion of macro 'C10_UNLIKELY_OR_CONST'
         if (C10_UNLIKELY_OR_CONST(!(cond))) {                                         \
             ^~~~~~~~~~~~~~~~~~~~~
      /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/c10/util/Exception.h:599:32: note: in expansion of macro 'TORCH_INTERNAL_ASSERT'
           C10_EXPAND_MSVC_WORKAROUND(TORCH_INTERNAL_ASSERT(cond, __VA_ARGS__)); \
                                      ^~~~~~~~~~~~~~~~~~~~~
      /root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc/scale_mask_softmax_wrapper.cpp:5:3: note: in expansion of macro 'AT_ASSERTM'
         AT_ASSERTM(x.type().is_cuda(), #x " must be a CUDA tensor")
         ^~~~~~~~~~
      /root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc/scale_mask_softmax_wrapper.cpp:17:3: note: in expansion of macro 'CHECK_CUDA'
         CHECK_CUDA(x);                                                               \
         ^~~~~~~~~~
      /root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc/scale_mask_softmax_wrapper.cpp:30:3: note: in expansion of macro 'CHECK_INPUT'
         CHECK_INPUT(real_seq_len);
         ^~~~~~~~~~~
      In file included from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/ATen/Tensor.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/ATen/Context.h:4,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/ATen/ATen.h:9,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
                       from /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/torch/extension.h:4,
                       from /root/gpt_exp/opt_colossal/EnergonAI-main/energonai/kernel/cuda_native/csrc/scale_mask_softmax_wrapper.cpp:2:
      /home/kg/anaconda3/lib/python3.7/site-packages/torch/include/ATen/core/TensorBody.h:194:30: note: declared here
         DeprecatedTypeProperties & type() const {
                                    ^~~~
      ninja: build stopped: subcommand failed.
      Traceback (most recent call last):
        File "/home/kg/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1723, in _run_ninja_build
          env=env)
        File "/home/kg/anaconda3/lib/python3.7/subprocess.py", line 487, in run
          output=stdout, stderr=stderr)
      subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

      The above exception was the direct cause of the following exception:

      Traceback (most recent call last):
        File "<string>", line 36, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/root/gpt_exp/opt_colossal/EnergonAI-main/setup.py", line 187, in <module>
          'console_scripts': ['energonai=energonai.cli:typer_click_object', ],
        File "/home/kg/anaconda3/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
          return distutils.core.setup(**attrs)
        File "/home/kg/anaconda3/lib/python3.7/distutils/core.py", line 148, in setup
          dist.run_commands()
        File "/home/kg/anaconda3/lib/python3.7/distutils/dist.py", line 966, in run_commands
          self.run_command(cmd)
        File "/home/kg/anaconda3/lib/python3.7/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/home/kg/anaconda3/lib/python3.7/site-packages/setuptools/command/install.py", line 68, in run
          return orig.install.run(self)
        File "/home/kg/anaconda3/lib/python3.7/distutils/command/install.py", line 545, in run
          self.run_command('build')
        File "/home/kg/anaconda3/lib/python3.7/distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/home/kg/anaconda3/lib/python3.7/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/home/kg/anaconda3/lib/python3.7/distutils/command/build.py", line 135, in run
          self.run_command(cmd_name)
        File "/home/kg/anaconda3/lib/python3.7/distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/home/kg/anaconda3/lib/python3.7/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/home/kg/anaconda3/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 79, in run
          _build_ext.run(self)
        File "/home/kg/anaconda3/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
          _build_ext.build_ext.run(self)
        File "/home/kg/anaconda3/lib/python3.7/distutils/command/build_ext.py", line 340, in run
          self.build_extensions()
        File "/home/kg/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 735, in build_extensions
          build_ext.build_extensions(self)
        File "/home/kg/anaconda3/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/home/kg/anaconda3/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
          self._build_extensions_serial()
        File "/home/kg/anaconda3/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
          self.build_extension(ext)
        File "/home/kg/anaconda3/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
          _build_ext.build_extension(self, ext)
        File "/home/kg/anaconda3/lib/python3.7/distutils/command/build_ext.py", line 534, in build_extension
          depends=ext.depends)
        File "/home/kg/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 565, in unix_wrap_ninja_compile
          with_cuda=with_cuda)
        File "/home/kg/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1404, in _write_ninja_file_and_compile_objects
          error_prefix='Error compiling objects for extension')
        File "/home/kg/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1733, in _run_ninja_build
          raise RuntimeError(message) from e
      RuntimeError: Error compiling objects for extension
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> energonai

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

I am using python 3.7.4 , torch 1.10.1+cu102, transformers 4.26.0, colossalai 0.2.5
@binmakeswell
Copy link
Member

Hi @0-1CxH We update the tested env here. Thanks.
https://github.com/hpcaitech/ColossalAI#Installation

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@0-1CxH @binmakeswell