Installing linking to clBlas

[nathan-sixnines]

I'm trying to build OpenCL-caffe and I'm running into a problem linking clBLAS:

[ 59%] Linking CXX executable caffe
../lib/libcaffe.so: undefined reference to `cblas_sgemv'
...

I've found some people online having a similar issue with regular Caffe, but this is a bit different because I want to use clBLAS for this version and not some other BLAS correct?

Is there something I need to put in the configuration for it to find the clBLAS?

I exported the folder where I compiled clBLAS as CLBLAS_ROOT. Is there something else I need to do for it to build?

Some outputs that might be useful:

-- Build files have been written to: /home/nathan/OpenCL-caffe/build
nathan@amdRig:/OpenCL-caffe/build$ make clean
nathan@amdRig:/OpenCL-caffe/build$ sudo cmake ..
-- Boost version: 1.46.1
-- Found the following Boost libraries:
-- system
-- thread
-- Found gflags (include: /usr/local/include, library: /usr/local/lib/libgflags.a)
-- Found glog (include: /usr/local/include, library: /usr/local/lib/libglog.so)
-- Found PROTOBUF Compiler: /usr/bin/protoc
-- Found lmdb (include: /usr/local/include, library: /usr/local/lib/liblmdb.so)
-- Found LevelDB (include: /usr/local/include, library: /usr/local/lib/libleveldb.so)
-- Found Snappy (include: /usr/include, library: /usr/lib/libsnappy.so)
-- Found OpenCL (include: /opt/AMDAPP/include, library: /usr/lib/libOpenCL.so)
-- Found clBLAS (include: /usr/local/include, library: /usr/local/lib64/libclBLAS.so)
-- OpenCV found (/usr/share/OpenCV)
-- NumPy ver. 1.7.1 found (include: /usr/local/lib/python2.7/dist-packages/numpy/core/include)
-- Boost version: 1.46.1
-- Found the following Boost libraries:
-- python

-- Detected Doxygen OUTPUT_DIRECTORY: ./doxygen/

-- ******************* Caffe Configuration Summary *******************
-- General:
-- Version : (Caffe doesn't declare its version in headers)
-- Git : aaa1c2c-dirty
-- System : Linux
-- C++ compiler : /usr/bin/c++
-- Release CXX flags : -O3 -DNDEBUG -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
-- Debug CXX flags : -g -fPIC -Wall -Wno-sign-compare -Wno-uninitialized

-- Build type : Release

-- BUILD_SHARED_LIBS : ON
-- BUILD_python : ON
-- BUILD_matlab : OFF
-- BUILD_docs : ON

-- CPU_ONLY : OFF

-- Dependencies:
-- BLAS : Yes (clBLAS)
-- Boost : Yes (ver. 1.46)
-- glog : Yes
-- gflags : Yes
-- protobuf : Yes (ver. 2.4.1)
-- lmdb : Yes (ver. 0.9.70)
-- Snappy : Yes (ver. 1.0.4)
-- LevelDB : Yes (ver. 1.9)
-- OpenCV : Yes (ver. 2.3.1)
-- OpenCL : Yes

-- clBLAS : Yes

-- Python:
-- Interpreter : /usr/bin/python2.7 (ver. 2.7.3)
-- Libraries : /usr/lib/libpython2.7.so (ver 2.7.3)

-- NumPy : /usr/local/lib/python2.7/dist-packages/numpy/core/include (ver 1.7.1)

-- Documentaion:
-- Doxygen : /usr/bin/doxygen (1.7.6.1)

-- config_file : /home/nathan/OpenCL-caffe/.Doxyfile

-- Install:

-- Install path : //home/nathan/OpenCL-caffe/build/install

-- Configuring done
CMake Warning at src/caffe/CMakeLists.txt:20 (add_library):
Cannot generate a safe linker search path for target caffe because files in
some directories may conflict with libraries in implicit directories:

link library [libprotobuf.so] in /usr/lib may be hidden by files in:
  /usr/local/lib

Some of these libraries may not be found correctly.

CMake Warning at src/caffe/CMakeLists.txt:20 (add_library):
Cannot generate a safe runtime search path for target caffe because files
in some directories may conflict with libraries in implicit directories:

runtime library [libprotobuf.so.7] in /usr/lib may be hidden by files in:
  /usr/local/lib

....

....
[ 53%] Building CXX object src/caffe/CMakeFiles/caffe.dir/util/insert_splits.cpp.o
[ 55%] Building CXX object src/caffe/CMakeFiles/caffe.dir/util/upgrade_proto.cpp.o
[ 55%] Building CXX object src/caffe/CMakeFiles/caffe.dir/util/ocl_wrapper.cpp.o
[ 56%] Building CXX object src/caffe/CMakeFiles/caffe.dir/internal_thread.cpp.o
[ 56%] Linking CXX shared library ../../lib/libcaffe.so
[ 58%] Built target caffe
Scanning dependencies of target caffe.bin
[ 59%] Building CXX object tools/CMakeFiles/caffe.bin.dir/caffe.cpp.o
[ 59%] Linking CXX executable caffe
../lib/libcaffe.so: undefined reference to cblas_sgemv' ../lib/libcaffe.so: undefined reference tocblas_dgemm'
../lib/libcaffe.so: undefined reference to cblas_sscal' ../lib/libcaffe.so: undefined reference tocblas_dgemv'
../lib/libcaffe.so: undefined reference to cblas_saxpy' ../lib/libcaffe.so: undefined reference tocblas_ddot'
../lib/libcaffe.so: undefined reference to cblas_dasum' ../lib/libcaffe.so: undefined reference tocblas_sgemm'
../lib/libcaffe.so: undefined reference to cblas_dscal' ../lib/libcaffe.so: undefined reference tocblas_scopy'
../lib/libcaffe.so: undefined reference to cblas_sasum' ../lib/libcaffe.so: undefined reference tocblas_daxpy'
../lib/libcaffe.so: undefined reference to cblas_dcopy' ../lib/libcaffe.so: undefined reference tocblas_sdot'
collect2: ld returned 1 exit status

[mpekalski]

I had the same issue yesterday. Still unsolved.

[mpekalski]

If you look at cmake/Dependenceis.cmake (lines 78-101, https://github.com/amd/OpenCL-caffe/blob/c46909b9543f52b3aea316093a1c72062d6777f9/cmake/Dependencies.cmake) it looks like clBLAS is not supported as a value for property BLAS. I do not understand why, as few lines before there is a piece of code for including clBLAS.

Maybe clBLAS and other versions of the library do not really cover the same scope of functions, or ATLAS or LAPACK cannot work with clBLAS.

lines 64-101

# ---[ clBLAS
find_package(clBLAS REQUIRED)
include_directories(SYSTEM ${CLBLAS_INCLUDE_DIRS})
list(APPEND Caffe_LINKER_LIBS ${CLBLAS_LIBRARIES})

# ---[ OpenCV
find_package(OpenCV QUIET COMPONENTS core highgui imgproc imgcodecs)
if(NOT OpenCV_FOUND) # if not OpenCV 3.x, then imgcodecs are not found
  find_package(OpenCV REQUIRED COMPONENTS core highgui imgproc)
endif()
include_directories(SYSTEM ${OpenCV_INCLUDE_DIRS})
list(APPEND Caffe_LINKER_LIBS ${OpenCV_LIBS})
message(STATUS "OpenCV found (${OpenCV_CONFIG_PATH})")

# ---[ BLAS
if(NOT APPLE)
  set(BLAS "Atlas" CACHE STRING "Selected BLAS library")
  set_property(CACHE BLAS PROPERTY STRINGS "Atlas;Open;MKL")

  if(BLAS STREQUAL "Atlas" OR BLAS STREQUAL "atlas")
    find_package(Atlas REQUIRED)
    include_directories(SYSTEM ${Atlas_INCLUDE_DIR})
    list(APPEND Caffe_LINKER_LIBS ${Atlas_LIBRARIES})
  elseif(BLAS STREQUAL "Open" OR BLAS STREQUAL "open")
    find_package(OpenBLAS REQUIRED)
    include_directories(SYSTEM ${OpenBLAS_INCLUDE_DIR})
    list(APPEND Caffe_LINKER_LIBS ${OpenBLAS_LIB})
  elseif(BLAS STREQUAL "MKL" OR BLAS STREQUAL "mkl")
    find_package(MKL REQUIRED)
    include_directories(SYSTEM ${MKL_INCLUDE_DIR})
    list(APPEND Caffe_LINKER_LIBS ${MKL_LIBRARIES})
    add_definitions(-DUSE_MKL)
  endif()
elseif(APPLE)
  find_package(vecLib REQUIRED)
  include_directories(SYSTEM ${vecLib_INCLUDE_DIR})
  list(APPEND Caffe_LINKER_LIBS ${vecLib_LINKER_LIBS})
endif()

[gujunli]

Hi @Aeium @mpekalski ,
Looks like Aeium's building is complaining there is no c based BLAS. It is not a clBLAS error. Do you have ATLAS or OpenBLAS installed? OpenCL-caffe requires both ATLAS (or OpenBLAS) and CLBLAS. On the configure file, you should specify which one (ATLAS or OpenBLAS) to use. cLBLAS is specified by env variable CLBLAS_ROOT.

[mpekalski]

@gujunli
I think that if he did not set the value of ${BLAS} to clBLAS in the summary above it would not write

-- BLAS : Yes (clBLAS)

as it does. But unless @Aeium gives us his full list of variables provided to cmake or attach CMakeCache.txt we will not know.

If you look at **Makefile.config.example* (https://github.com/amd/OpenCL-caffe/blob/c46909b9543f52b3aea316093a1c72062d6777f9/Makefile.config.example) lines 29-31 it clearly says you cannot provide clBLAS as a value for BLAS

# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := atlas

Another issue, I do not understand, is why LAPACK does not support clBLAS but does ACML, see FindLAPACK.cmake (https://github.com/amd/OpenCL-caffe/blob/c46909b9543f52b3aea316093a1c72062d6777f9/cmake/Modules/FindLAPACK.cmake)

[nathan-sixnines]

Well, I'm past the issue.

When I was getting that error I don't think I had Atlas or some BLAS other than clBLAS installed.

Ultimately what happened was I rm -rf'd the wrong directory and decided it would be best to just start over on Ubuntu 14.04 instead of 12.04. I just made sure to get Atlas this time.

[hughperkins]

decided it would be best to just start over on Ubuntu 14.04 instead of 12.04.

Not a fan of Unity I take it :-P

[nathan-sixnines]

Ok, well OpenCL-caffe actually built 100%, but then it had a segmentation fault on the runtest. I decided to try again with clBLAS' precompiled binaries, but now i'm having trouble linking to them again.

I have been trying to edit the file in cmake under modules to ensure the cmake script gets the correct files, but I'm running into an issue where the cmake script seems blind to most of my environment variables.

Turns out the sudo shell has a different set of environment variables, so that nixes my CLBLASROOT variable I was using in the cmake script, because "sudo cmake .." would not have that variable in the new root shell it makes to execute in, or something like that.

It seems like the implication of this is that using sudo cmake and cmake are actually more different than I thought, and I need a different strategy other than exporting environment variables if I want to link things to this program if I need to use sudo cmake, which I think I do.

Anyway, that's my progress report so far. Maybe I would be a fan of unity if I had more experience with Linux, which clearly would help out here anyway.

[gujunli]

Thanks for the update. It seems like you are having some inconsistent setting in your env variables. Clblas compiling can be with errors as well. Sometimes start all over with a clean environment can help. If still errors, post it here and also clblas github issues.
Thanks.
Junli

Sent from my iPhone

On Dec 30, 2015, at 10:46 PM, Aeium [email protected] wrote:

Ok, well OpenCL-caffe actually built 100%, but then it had a segmentation fault on the runtest. I decided to try again with clBLAS' precompiled binaries, but now i'm having trouble linking to them again.

I have been trying to edit the file in cmake under modules to ensure the cmake script finds clBLAS properly, but I'm running into an issue where the cmake script seems blind to most of my environment variables.

Turns out the sudo shell has a different set of environment variables, so that nixes my CLBLASROOT variable I was using in the cmake script, because "sudo cmake .." would not have that variable in the new root shell it makes to execute in, or something like that.

It seems like the implication of this is that using sudo cmake and cmake are actually more different than I thought, and I need a different strategy other than exporting environment variables if I want to link things to this program if I need to use sudo cmake, which I think I do.

Anyway, that's my progress report so far. Maybe I would be a fan of unity if I had more experience with Linux, which clearly would help out here anyway.

—
Reply to this email directly or view it on GitHub.