Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can you give me some advises on makefile? #51

Open
ZhenhuiTang opened this issue May 31, 2020 · 18 comments
Open

Can you give me some advises on makefile? #51

ZhenhuiTang opened this issue May 31, 2020 · 18 comments

Comments

@ZhenhuiTang
Copy link

I encounter some difficulties during make makefile, as follow:

make: Circular tf_nndistance.cu <- tf_nndistance.cu.o dependency dropped.
g++ tf_nndistance.cpp tf_nndistance.cu.o -o tf_nndistance_so.so \
-I /usr/local/cuda-10.0/include/ -I /home/machinelearning/.local/lib/python3.6/site-packages/tensorflow_core/include -L /usr/local/cuda-10.0/lib64/ -lcudart -L /home/machinelearning/.local/lib/python3.6/site-packages/tensorflow_core -ltensorflow_framework \
    -shared -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11 -fPIC -O2
/usr/bin/ld: cannot find -ltensorflow_framework
collect2: error: ld returned 1 exit status
makefile:13: recipe for target 'tf_nndistance_so.so' failed
make: *** [tf_nndistance_so.so] Error 1

The environment is tf.version == 1.15 and cuda == 10.0, and the modified content as follow:

cuda_inc = /usr/local/cuda-10.0/include/
cuda_lib = /usr/local/cuda-10.0/lib64/
nvcc = /usr/local/cuda-10.0/bin/nvcc
tf_inc = /home/machinelearning/.local/lib/python3.6/site-packages/tensorflow_core/include
tf_lib =/home/machinelearning/.local/lib/python3.6/site-packages/tensorflow_core

Thanks a lot.

@TerenceCYJ
Copy link

Hi. The same problem appears. Have you got the solution?

@TerenceCYJ
Copy link

Hi. The same problem appears. Have you got the solution?

This can be solved by:
cd /home/machinelearning/.local/lib/python3.6/site-packages/tensorflow_core
ln -s libtensorflow_framework.so.1 libtensorflow_framework.so

@GopiRajuMatta
Copy link

@TerenceCYJ

Thank you.

I've executed same commands, but problem is not solved.
Can you please once look into the same.

@TerenceCYJ
Copy link

@TerenceCYJ

Thank you.

I've executed same commands, but problem is not solved.
Can you please once look into the same.

Hi, @GopiRajuMatta. I just fix the bug by what I wrote. And my environment is tf.version == 1.15 and cuda == 10.0.

@GopiRajuMatta
Copy link

Hi, @TerenceCYJ

Thank you Terence, It was built without any error. But when I try to run demo.py, it is throwing this error.

"
Traceback (most recent call last):
File "demo.py", line 32, in
model_module = importlib.import_module('.%s' % args.model_type, 'models')
File "/home/gopi/anaconda3/envs/pcn_test2/lib/python3.6/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 994, in _gcd_import
File "", line 971, in _find_and_load
File "", line 955, in _find_and_load_unlocked
File "", line 665, in _load_unlocked
File "", line 678, in exec_module
File "", line 219, in _call_with_frames_removed
File "/home/gopi/git/pcn/models/pcn_cd.py", line 4, in
from tf_util import *
File "/home/gopi/git/pcn/tf_util.py", line 4, in
from pc_distance import tf_nndistance, tf_approxmatch
File "/home/gopi/git/pcn/pc_distance/tf_nndistance.py", line 5, in
nn_distance_module=tf.load_op_library(os.path.join(BASE_DIR, 'tf_nndistance_so.so'))
File "/home/gopi/anaconda3/envs/pcn_test2/lib/python3.6/site-packages/tensorflow_core/python/framework/load_library.py", line 61, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
tensorflow.python.framework.errors_impl.NotFoundError: /home/gopi/git/pcn/pc_distance/tf_nndistance_so.so: undefined symbol: _ZN10tensorflow12OpDefBuilder5InputESs
"

Any idea, where it went wrong?

Thank you
Gopi

@TerenceCYJ
Copy link

Hi, Gopi. I think you should check your pc_distance.
After successful make, there should be a tf_nndistance_so.so
In my setting, after make, ls pc_distance
Get:__init__.py __pycache__ tf_approxmatch.cu tf_approxmatch.py tf_nndistance.cpp tf_nndistance.cu.o tf_nndistance_so.so makefile tf_approxmatch.cpp tf_approxmatch.cu.o tf_approxmatch_so.so tf_nndistance.cu tf_nndistance.py

@GopiRajuMatta
Copy link

Thank you Terence.

These are the files after make, it has all the files as you reported.
"init_.py tf_approxmatch.cpp tf_approxmatch.py tf_nndistance.cu tf_nndistance_so.so makefile tf_approxmatch.cu tf_approxmatch_so.so tf_nndistance.cu.o pycache tf_approxmatch.cu.o tf_nndistance.cpp tf_nndistance.py"

However, I am getting the reported error.

Thank you
Gopi

@GopiRajuMatta
Copy link

Hi Terence,

It seems, problem is with file tf_nndistance_so.so. It was compiled without errors. But I am not sure, why I am getting that particular error. I have used cuda=9.0, tf.version == 1.15. Is there any problem with this? If you don't mind, can you please share with me your compiled pc_distance folder.

Thank you
Gopi

@mihaimorariu
Copy link
Contributor

mihaimorariu commented Jul 4, 2020

I encountered the same issues. After several days of bashing my head against the desk, I got the code up and running using the following steps:

  1. Create a conda environment with cuda=9.2, tf=1.12, open3d=0.8.0.0 and python=3.6:
conda create -n pcn tensorflow=1.12 python=3.6
conda activate pcn
conda install -c conda-forge nvcc_linux-64=9.2
conda install -c nvidia cudatoolkit=9.2
conda install -c open3d-admin open3d=0.8
  1. Modify makefile as follows (replace user with your username):
cuda_inc = /home/user/.conda/envs/pcn/include/
cuda_lib = /home/user/.conda/envs/pcn/lib/
nvcc     = /home/user/.conda/envs/pcn/bin/nvcc
tf_inc   = /home/user/.conda/envs/pcn/lib/python3.6/site-packages/tensorflow/include
tf_lib   = /home/user/.conda/envs/pcn/lib/python3.6/site-packages/tensorflow

all: tf_nndistance_so.so tf_approxmatch_so.so

tf_nndistance.cu.o: tf_nndistance.cu
	$(nvcc) tf_nndistance.cu -o tf_nndistance.cu.o -c -O2 -DGOOGLE_CUDA=1 -x cu -Xcompiler -fPIC

tf_nndistance_so.so: tf_nndistance.cpp tf_nndistance.cu.o
	g++ tf_nndistance.cpp tf_nndistance.cu.o -o tf_nndistance_so.so -shared -std=c++11 -fPIC -O2 \
            -I $(tf_inc) -D_GLIBCXX_USE_CXX11_ABI=1 -L $(tf_lib) -ltensorflow_framework -L $(cuda_lib) -lcudart

tf_approxmatch.cu.o: tf_approxmatch.cu
	$(nvcc) tf_approxmatch.cu -o tf_approxmatch.cu.o -c -O2 -DGOOGLE_CUDA=1 -x cu -Xcompiler -fPIC

tf_approxmatch_so.so: tf_approxmatch.cpp tf_approxmatch.cu.o
	g++ -shared $(CPPFLAGS) tf_approxmatch.cpp tf_approxmatch.cu.o -o tf_approxmatch_so.so -shared -std=c++11 -fPIC -O2 \
            -I $(tf_inc) -D_GLIBCXX_USE_CXX11_ABI=1 -L $(tf_lib) -ltensorflow_framework -L $(cuda_lib) -lcudart

clean:
	rm -rf *.o *.so
  1. Run make clean && make inside the conda environment.

Hope this helps!

@GopiRajuMatta
Copy link

@mihaimorariu, I really appreciate your help.
I am getting new errors. I will work around. I will try to resolve those.
I will update you.

Thank you so much
Gopi

@francisduan
Copy link

Hi I am having exactly same error when running demo.py. Have you figured it out @GopiRajuMatta ?

@GopiRajuMatta
Copy link

Hi I am having exactly same error when running demo.py. Have you figured it out @GopiRajuMatta ?

Hello @francisduan

Presently, I am working on other problem.
I will try again and let you know, if it is resolved.

@mihaimorariu would be able to help you.

Thank you
Gopi

@StefanosPapavasileiou
Copy link

StefanosPapavasileiou commented Dec 21, 2020

Hello guys,

Has anyone of you resolved the ''make: Circular tf_nndistance.cu <- tf_nndistance.cu.o dependency dropped.'' problem mentioned above? I have tried pretty much anything i could find and no result.
I am working with tf .version==1.12, cuda== 9.0
Thanks in advance,
Yakamoko

@gillbam
Copy link

gillbam commented Jan 1, 2021

Hi guys, removing -D_GLIBCXX_USE_CXX11_ABI=1 solved my probelm

@StefanosPapavasileiou
Copy link

Hi gillbam,

Could you plz define the version of tf and cuda-gcc you are using in order to achieve this?

Furthermore, the code you are mentioning for removal is on the makefile right?

Thanks in advance,
Yakamoko

@gillbam
Copy link

gillbam commented Jan 5, 2021

Hi gillbam,

Could you plz define the version of tf and cuda-gcc you are using in order to achieve this?

Furthermore, the code you are mentioning for removal is on the makefile right?

Thanks in advance,
Yakamoko

Hi Yakamoko,
I have tf 1.12, cuda 10.1 installed. Check my attached code.

cuda_inc = /usr/local/cuda-10.1/include/
cuda_lib = /usr/local/cuda-10.1/lib64/
nvcc = /usr/local/cuda-10.1/bin/nvcc
tf_inc = /root/anaconda3/envs/pcn/lib/python3.6/site-packages/tensorflow/include
tf_lib = /root/anaconda3/envs/pcn/lib/python3.6/site-packages/tensorflow

all: tf_nndistance_so.so tf_approxmatch_so.so

tf_nndistance.cu.o: tf_nndistance.cu
   $(nvcc) tf_nndistance.cu -o tf_nndistance.cu.o -c -O2 -DGOOGLE_CUDA=1 -x cu -Xcompiler -fPIC

tf_nndistance_so.so: tf_nndistance.cpp tf_nndistance.cu.o
   g++ tf_nndistance.cpp tf_nndistance.cu.o -o tf_nndistance_so.so
   -I $(cuda_inc) -I $(tf_inc) -L $(cuda_lib) -lcudart -L $(tf_lib) -ltensorflow_framework
   -shared -std=c++11 -fPIC -O2

tf_approxmatch.cu.o: tf_approxmatch.cu
   $(nvcc) tf_approxmatch.cu -o tf_approxmatch.cu.o -c -O2 -DGOOGLE_CUDA=1 -x cu -Xcompiler -fPIC

tf_approxmatch_so.so: tf_approxmatch.cpp tf_approxmatch.cu.o
    g++ -shared $(CPPFLAGS) tf_approxmatch.cpp tf_approxmatch.cu.o -o tf_approxmatch_so.so
    -I $(cuda_inc) -I $(tf_inc) -L $(cuda_lib) -lcudart -L $(tf_lib) -ltensorflow_framework
    -shared -std=c++11 -fPIC -O2

clean:
    rm -rf *.o *.so

@gzach1
Copy link

gzach1 commented Nov 12, 2021

Hi, I am getting the same error:

make: Circular tf_nndistance.cu <- tf_nndistance.cu.o dependency dropped.
g++ tf_nndistance.cpp tf_nndistance.cu.o -o tf_nndistance_so.so
....
make (e=2): The system cannot find the file specified.
make: *** [makefile:13: tf_nndistance_so.so] Error 2

and I have intalled:
conda install -c nvidia/label/cuda-11.3.0 cuda-nvcc and
conda install -c nvidia cudatoolkit=9.2

using the same makefile above. Is it a version problem still?
Thanks in advance.

@MarioCavero
Copy link

I am trying to run demo.py with the right versions and with the updated versions, and I keep getting an error that I cannot fix. I've been working around this for some days already and can't manage to get anything better.

Traceback (most recent call last):
  File "demo.py", line 121, in <module>
    main()
  File "demo.py", line 77, in main
    model_module = importlib.import_module('.%s' % args.model_type, 'models')
  File "/home/mk/anaconda3/envs/pcnconda/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/mk/Downloads/pcn-master/models/pcn_cd.py", line 26, in <module>
    from tf_util import *
  File "/home/mk/Downloads/pcn-master/tf_util.py", line 26, in <module>
    from pc_distance import tf_nndistance, tf_approxmatch
  File "/home/mk/Downloads/pcn-master/pc_distance/tf_approxmatch.py", line 21, in <module>
    @ops.RegisterShape('ApproxMatch')
AttributeError: module 'tensorflow.python.framework.ops' has no attribute 'RegisterShape'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants