linux下用cmake对caffe静态编译时-static-libstdc++参数无效的问题
2017-06-12 17:50
531 查看
以下是用于cmake 生成 Makefile文件对Caffe进行静态库连接编译的shell脚本,
在脚本中,调用cmake生成Makefile时,添加了
这个问题困扰了几天,后来通过比较
如下是caffe.bin项目link.txt(./tools/CMakeFiles/caffe.bin.dir/link.txt),太长我加了分行符:
就是这尾部的
静态编译后,ldd 查看bin下的caffe依赖关系显示如下,除了系统库之外,不再依赖任何其他动态库,文件大小也达到28MB。
[hadoop@t2-centos6 bin]$ ldd ../release/caffe-ssd_linux_x86_64/bin/caffe
linux-vdso.so.1 => (0x00007fff8fb58000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003c24600000)
librt.so.1 => /lib64/librt.so.1 (0x0000003c24e00000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000003c24a00000)
libm.so.6 => /lib64/libm.so.6 (0x0000003c25200000)
libc.so.6 => /lib64/libc.so.6 (0x0000003c24200000)
/lib64/ld-linux-x86-64.so.2 (0x0000003c23e00000)
但是为什么opencv的库会导致这个问题。一直没搞明白。
本文贴出的脚本并不完整
关于caffe静态编译的完整脚本,请从从csdn CODE获取:
https://code.csdn.net/10km/caffe-static
#!/bin/bash # cmake 静态编译 caffe-ssd 代码脚本 # author guyadong@gdface.net shell_folder=$(cd "$(dirname "$0")";pwd) . $shell_folder/build_funs . $shell_folder/build_vars # 项目安装路径 install_path=$SSD_INSTALL_PATH echo install_path:$install_path # gflags 安装路径 gflags_root=$GFLAGS_INSTALL_PATH exit_if_not_exist "$gflags_root/lib/libgflags.a" "not found $gflags_root/lib/libgflags.a,please build $GFLAGS_PREFIX" # glog 安装路径 glog_root=$GLOG_INSTALL_PATH exit_if_not_exist "$glog_root/lib/libglog.a" "not found $glog_root/lib/libglog.a,please build $GLOG_PREFIX" # hdf5 cmake 位置 hdf5_cmake_dir=$HDF5_INSTALL_PATH/share/cmake exit_if_not_exist $hdf5_cmake_dir "not found $hdf5_cmake_dir,please build $HDF5_PREFIX" exit_if_not_exist $BOOST_INSTALL_PATH "not found $BOOST_INSTALL_PATH,please build $BOOST_PREFIX" exit_if_not_exist $OPENBLAS_INSTALL_PATH "not found $OPENBLAS_INSTALL_PATH,please build $OPENBLAS_PREFIX" exit_if_not_exist $PROTOBUF_INSTALL_PATH "not found $PROTOBUF_INSTALL_PATH,please build $PROTOBUF_PREFIX" # protobuf lib 路径, # centos下安装路径可能是lib64 [ -e "$PROTOBUF_INSTALL_PATH/lib" ] && protobuf_lib=$PROTOBUF_INSTALL_PATH/lib [ -e "$PROTOBUF_INSTALL_PATH/lib64" ] && protobuf_lib=$PROTOBUF_INSTALL_PATH/lib64 exit_if_not_exist $SNAPPY_INSTALL_PATH "not found $SNAPPY_INSTALL_PATH,please build $SNAPPY_PREFIX" # lmdb 安装路径根目录 lmdb_install_root=$LMDB_INSTALL_PATH/usr/local exit_if_not_exist $lmdb_install_root "not found $lmdb_install_root,please build lmdb" exit_if_not_exist $LEVELDB_INSTALL_PATH "not found $LEVELDB_INSTALL_PATH,please build $LEVELDB_PREFIX" # opencv 配置文件(OpenCVConfig.cmake)所在路径 opencv_cmake_dir=$OPENCV_INSTALL_PATH/share/OpenCV exit_if_not_exist $opencv_cmake_dir "not found $opencv_cmake_dir,please build $OPENCV_PREFIX" pushd $SOURCE_ROOT/$SSD_FOLDER clean_folder build.gcc #mkdir_if_not_exist build.gcc pushd build.gcc # 指定 OpenBLAS 安装路径 参见 $caffe_source/cmake/Modules/FindOpenBLAS.cmake export OpenBLAS_HOME=$OPENBLAS_INSTALL_PATH # 指定 lmdb 安装路径 参见 $caffe_source/cmake/Modules/FindLMDB.cmake.cmake export LMDB_DIR=$lmdb_install_root # 指定 leveldb 安装路径 参见 $caffe_source/cmake/Modules/FindLevelDB.cmake.cmake export LEVELDB_ROOT=$LEVELDB_INSTALL_PATH # GLOG_ROOT_DIR 参见 $caffe_source/cmake/Modules/FindGlog.cmake # GFLAGS_ROOT_DIR 参见 $caffe_source/cmake/Modules/FindGFlags.cmake # HDF5_ROOT 参见 https://cmake.org/cmake/help/v3.8/module/FindHDF5.html # BOOST_ROOT,Boost_NO_SYSTEM_PATHS 参见 https://cmake.org/cmake/help/v3.8/module/FindBoost.html # SNAPPY_ROOT_DIR 参见 $caffe_source/cmake/Modules/FindSnappy.cmake # PROTOBUF_LIBRARY,PROTOBUF_PROTOC_LIBRARY... 参见 https://cmake.org/cmake/help/v3.8/module/FindProtobuf.html # OpenCV_DIR 参见https://cmake.org/cmake/help/v3.8/command/find_package.html $CMAKE_EXE "$(dirs +1)" $CMAKE_VARS_DEFINE -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=$install_path \ -DCMAKE_EXE_LINKER_FLAGS="-static-libstdc++ -static-libgcc" \ -DGLOG_ROOT_DIR=$glog_root \ -DGFLAGS_ROOT_DIR=$gflags_root \ -DHDF5_ROOT=$HDF5_INSTALL_PATH \ -DBOOST_ROOT=$BOOST_INSTALL_PATH \ -DBoost_NO_SYSTEM_PATHS=on \ -DSNAPPY_ROOT_DIR=$SNAPPY_INSTALL_PATH \ -DOpenCV_DIR=$opencv_cmake_dir \ -DPROTOBUF_LIBRARY=$protobuf_lib/libprotobuf.a \ -DPROTOBUF_PROTOC_LIBRARY=$protobuf_lib/libprotoc.a \ -DPROTOBUF_LITE_LIBRARY=$protobuf_lib/libprotobuf-lite.a \ -DPROTOBUF_PROTOC_EXECUTABLE=$PROTOBUF_INSTALL_PATH/bin/protoc \ -DPROTOBUF_INCLUDE_DIR=$PROTOBUF_INSTALL_PATH/include \ -DCPU_ONLY=ON \ -DBLAS=Open \ -DBUILD_SHARED_LIBS=off \ -DBUILD_docs=off \ -DBUILD_python=off \ -DBUILD_python_layer=off \ -DUSE_LEVELDB=on \ -DUSE_LMDB=on \ -DUSE_OPENCV=on exit_on_error remove_if_exist $install_path make -j $MAKE_JOBS install exit_on_error popd rm -fr build.gcc popd
在脚本中,调用cmake生成Makefile时,添加了
-DCMAKE_EXE_LINKER_FLAGS="-static-libstdc++ -static-libgcc"参数用于指定 将libstdc++,libgcc以静态库形式连接,然而在实际使用过程发现当指定
USE_OPENCV=on(即使用OpenCV,opencv已经预先做了静态库编译)时,编译出的caffe,用ldd查看还是会依赖libstdc++.so和libgcc.so,也就是说
-static-libstdc++ -static-libgcc无效了。。。
USE_OPENCV=off时则能正常静态连接。
这个问题困扰了几天,后来通过比较
<project>.dir下的link.txt(cmake生成的),发现,当
USE_OPENCV=on时生成的link.txt中,自动在opencv静态库加了-lstdc++ 参数。。。
如下是caffe.bin项目link.txt(./tools/CMakeFiles/caffe.bin.dir/link.txt),太长我加了分行符:
/usr/local/bin/g++ -fPIC -Wall -Wno-sign-compare -Wno-uninitialized -O3 -DNDEBUG -static-libstdc++ -static-libgcc -rdynamic CMakeFiles/caffe.bin.dir/caffe.cpp.o -o caffe -Wl,-rpath,:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -Wl,--whole-archive ../lib/libcaffe.a -Wl,--no-whole-archive ../lib/libproto.a /home/hadoop/tmp/dl/depends/release/boost_linux_x86_64/lib/libboost_system.a /home/hadoop/tmp/dl/depends/release/boost_linux_x86_64/lib/libboost_thread.a /home/hadoop/tmp/dl/depends/release/boost_linux_x86_64/lib/libboost_filesystem.a /home/hadoop/tmp/dl/depends/release/boost_linux_x86_64/lib/libboost_regex.a -lpthread /home/hadoop/tmp/dl/depends/release/glog_linux_x86_64/lib/libglog.a /home/hadoop/tmp/dl/depends/release/gflags_linux_x86_64/lib/libgflags.a /home/hadoop/tmp/dl/depends/release/protobuf_linux_x86_64/lib64/libprotobuf.a -lpthread /home/hadoop/tmp/dl/depends/release/glog_linux_x86_64/lib/libglog.a /home/hadoop/tmp/dl/depends/release/gflags_linux_x86_64/lib/libgflags.a /home/hadoop/tmp/dl/depends/release/protobuf_linux_x86_64/lib64/libprotobuf.a /home/hadoop/tmp/dl/depends/release/hdf5_linux_x86_64/lib/libhdf5_cpp.a /home/hadoop/tmp/dl/depends/release/hdf5_linux_x86_64/lib/libhdf5.a /home/hadoop/tmp/dl/depends/release/hdf5_linux_x86_64/lib/libhdf5_hl_cpp.a /home/hadoop/tmp/dl/depends/release/hdf5_linux_x86_64/lib/libhdf5_hl.a /home/hadoop/tmp/dl/depends/release/hdf5_linux_x86_64/lib/libhdf5_cpp.a /home/hadoop/tmp/dl/depends/release/hdf5_linux_x86_64/lib/libhdf5.a /home/hadoop/tmp/dl/depends/release/hdf5_linux_x86_64/lib/libhdf5_hl_cpp.a /home/hadoop/tmp/dl/depends/release/hdf5_linux_x86_64/lib/libhdf5_hl.a /home/hadoop/tmp/dl/depends/release/lmdb_linux_x86_64/usr/local/lib/liblmdb.a /home/hadoop/tmp/dl/depends/release/leveldb_linux_x86_64/lib/libleveldb.a /home/hadoop/tmp/dl/depends/release/snappy_linux_x86_64/lib/libsnappy.a /home/hadoop/tmp/dl/depends/release/opencv_linux_x86_64/lib/libopencv_highgui.a /home/hadoop/tmp/dl/depends/release/opencv_linux_x86_64/share/OpenCV/3rdparty/lib/liblibjpeg.a /home/hadoop/tmp/dl/depends/release/opencv_linux_x86_64/share/OpenCV/3rdparty/lib/liblibpng.a /home/hadoop/tmp/dl/depends/release/opencv_linux_x86_64/share/OpenCV/3rdparty/lib/liblibtiff.a /home/hadoop/tmp/dl/depends/release/opencv_linux_x86_64/share/OpenCV/3rdparty/lib/liblibjasper.a /home/hadoop/tmp/dl/depends/release/opencv_linux_x86_64/share/OpenCV/3rdparty/lib/libIlmImf.a /home/hadoop/tmp/dl/depends/release/opencv_linux_x86_64/lib/libopencv_imgproc.a /home/hadoop/tmp/dl/depends/release/opencv_linux_x86_64/lib/libopencv_core.a /home/hadoop/tmp/dl/depends/release/opencv_linux_x86_64/share/OpenCV/3rdparty/lib/libzlib.a -lstdc++ -lm -lpthread -lrt /home/hadoop/tmp/dl/depends/release/OpenBLAS_linux_x86_64/lib/libopenblas.a -ldl -pthread -lrt
就是这尾部的
-lstdc++参数导致
-static-libstdc++ -static-libgcc无效,尝试手工删除
-lstdc++,则编译通过,stdc++,libgcc都能静态连接进来了。于是果然在cmake生成Makefile后,添加了如下代码,则问题解决:
# 修改所有 link.txt 删除-lstdc++ 选项,保证静态连接libstdc++库,否则在USE_OPENCV=on的情况下,libstdc++不会静态链接 for file in $(find . -name link.txt) do echo "modifing file: $file" #sed -i -r "s/-lstdc\+\+/ /g" $file done
静态编译后,ldd 查看bin下的caffe依赖关系显示如下,除了系统库之外,不再依赖任何其他动态库,文件大小也达到28MB。
[hadoop@t2-centos6 bin]$ ldd ../release/caffe-ssd_linux_x86_64/bin/caffe
linux-vdso.so.1 => (0x00007fff8fb58000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003c24600000)
librt.so.1 => /lib64/librt.so.1 (0x0000003c24e00000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000003c24a00000)
libm.so.6 => /lib64/libm.so.6 (0x0000003c25200000)
libc.so.6 => /lib64/libc.so.6 (0x0000003c24200000)
/lib64/ld-linux-x86-64.so.2 (0x0000003c23e00000)
但是为什么opencv的库会导致这个问题。一直没搞明白。
本文贴出的脚本并不完整
关于caffe静态编译的完整脚本,请从从csdn CODE获取:
https://code.csdn.net/10km/caffe-static
相关文章推荐
- caffe 问题集锦之使用cmake编译多GPU时,更改USE_NCCL=1无效
- caffe 问题集锦之使用cmake编译多GPU时,更改USE_NCCL=1无效
- CMake的交叉编译问题(Linux x86 - Linux arm11)
- Linux下常见的~/.bashrc、/etc/profile、/etc/ld.so.config小科普以及caffe编译遇到的相关问题解决
- C语言学习4: 函数返回值与传入参数,关于函数值传递和类型隐性转换,变量不同的作用域,static变量,多文件编译例如两个C文件,显示函数调用语句跳转,递归,斐波那契数列,多文件编译相同变量的问题。
- CMake的交叉编译问题(Linux x86 - Linux arm11)
- gcc编译参数的顺序问题——关于static和动态链接库
- [caffe]:关于调用caffe库,cmake 编译问题
- Ubuntu16.04+anaconda2+caffe+ssd+opencv3.1.0在编译caffe过程中的问题及解决方法 主要遇到三个问题,前两个是caffe在cmake过程中的问题,后一
- linux下编译静态(static)库
- CMake的交叉编译问题(Linux x86 - Linux arm11)
- CMake的交叉编译问题(Linux x86 - Linux arm11)
- [链接] Linux下常见的~/.bashrc、/etc/profile、/etc/ld.so.config小科普以及caffe编译遇到的相关问题解决
- VC6静态编译的一个问题
- linux下request.getInputStream()接收的参数为空,windows下没有这个问题,为什么?
- apache 问题--提供了一个无效的参数!
- vc.net中MFC静态与动态链接问题与release方式编译
- 编译内核(linux-2.4.19)时遇到的问题-make dep
- .NET中回发或回调参数无效问题的解决
- Asp.net2.0回发或回调参数无效问题的解决