菜鸟跑通caffe（四）——fast-cnn编译记录

问题：CUDA 由6.5升级到7.5导致原有的caffe和digits不在可用。


https://github.com/BVLC/caffe/issues/2808备注，原有的caffe和digits对应6.5，由nvidia caffe v2版本copy得来，对应的cudnn也是v2。现在：参考：http://www.tuicool.com/articles/FfaERfB https://github.com/rbgirshick/fast-rcnn1）copy fast-rcnn的github文件。

git clone --recursive https://github.com/rbgirshick/fast-rcnn.git

2）安装和验证python组件包




3）Cython模块
     cd $FRCN_ROOT/lib     make
4）生成caffe和pycaffe

cd $FRCN_ROOT/caffe-fast-rcnn

cat > make.log

make -j8 2>&1|tee make.log

make pycaffe  2>&1|tee make.log

对应的Makefile.config如下：

de >## Refer to http://caffe.berkeleyvision.org/installation.htmlde> de ># Contributions simplifying and improving our build system are welcome!de>
de >de>
de ># cuDNN acceleration switch (uncomment to build with cuDNN).de>
de >USE_CUDNN := 1de>
de >de>
de ># CPU-only switch (uncomment to build without GPU support).de>
de ># CPU
4000
_ONLY := 1de>
de >de>
de ># To customize your choice of compiler, uncomment and set the following.de>
de ># N.B. the default for Linux is g++ and the default for OSX is clang++de>
de >CUSTOM_CXX := g++de>
de >de>
de ># CUDA directory contains bin/ and lib/ directories that we need.de>
de >CUDA_DIR := /usr/local/cudade>
de ># On Ubuntu 14.04, if cuda tools are installed viade>
de ># "sudo apt-get install nvidia-cuda-toolkit" then use this instead:de>
de ># CUDA_DIR := /usrde>
de >de>
de ># CUDA architecture setting: going with all of them.de>
de ># For CUDA < 6.0, comment the *_50 lines for compatibility.de>
de >CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \de>
de > -gencode arch=compute_20,code=sm_21 \de>
de > -gencode arch=compute_30,code=sm_30 \de>
de > -gencode arch=compute_35,code=sm_35 \de>
de > -gencode arch=compute_50,code=sm_50 \de>
de > -gencode arch=compute_50,code=compute_50de>
de >de>
de ># BLAS choice:de>
de ># atlas for ATLAS (default)de>
de ># mkl for MKLde>
de ># open for OpenBlasde>
de >BLAS := atlasde>
de ># Custom (MKL/ATLAS/OpenBLAS) include and lib directories.de>
de ># Leave commented to accept the defaults for your choice of BLASde>
de ># (which should work)!de>
de >BLAS_INCLUDE := /usr/includede>
de >BLAS_LIB := /usr/libde>
de >de>
de ># This is required only if you will compile the matlab interface.de>
de ># MATLAB directory should contain the mex binary in /bin.de>
de >MATLAB_DIR := /usr/local/MATLAB/R2014bde>
de ># MATLAB_DIR := /Applications/MATLAB_R2012b.appde>
de >de>
de ># NOTE: this is required only if you will compile the python interface.de>
de ># We need to be able to find Python.h and numpy/arrayobject.h.de>
de >PYTHON_INCLUDE := /usr/include/python2.7 \de>
de > /usr/lib/python2.7/dist-packages/numpy/core/includede>
de ># Anaconda Python distribution is quite popular. Include path:de>
de ># Verify anaconda location, sometimes it's in root.de>
de ># ANACONDA_HOME := $(HOME)/anacondade>
de ># PYTHON_INCLUDE := $(ANACONDA_HOME)/include \de>
de > # $(ANACONDA_HOME)/include/python2.7 \de>
de > # $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \de>
de >de>
de ># We need to be able to find libpythonX.X.so or .dylib.de>
de >PYTHON_LIB := /usr/libde>
de ># PYTHON_LIB := $(ANACONDA_HOME)/libde>
de >de>
de ># Uncomment to support layers written in Python (will link against Python libs)de>
de ># This will require an additional dependency boost_regex provided by boost.de>
de >WITH_PYTHON_LAYER := 1de>
de >de>
de ># Whatever else you find you need goes here.de>
de >INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/includede>
de >LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/libde>
de >de>
de ># Uncomment to use `pkg-config` to specify OpenCV library paths.de>
de ># (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)de>
de >USE_PKG_CONFIG := 1de>
de >de>
de >BUILD_DIR := buildde>
de >DISTRIBUTE_DIR := distributede>
de >de>
de ># Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171de> de >DEBUG := 1de>
de >de>
de ># The ID of the GPU that 'make runtest' will use to run unit tests.de>
de >TEST_GPUID := 0de>
de >de>
de ># enable pretty build (comment to see full commands)de>
de >Q ?= @de>
出现问题：

de >...de>
de >de>
de >NVCC src/caffe/layers/deconv_layer.cude>
de >NVCC src/caffe/layers/cudnn_conv_layer.cude>
de >src/caffe/layers/cudnn_conv_layer.cu(81): error: argument of type "cudnnAddMode_t" is incompatible with parameter of type "const void *"de>
de > detected during instantiation of "void caffe::CuDNNConvolutionLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" de>
de >(157): herede>
de >de>
de >...de>
de >de>
de >20 errors detected in the compilation of "/tmp/tmpxft_00002703_00000000-16_cudnn_conv_layer.compute_50.cpp1.ii".de>
de >make: *** [.build_release/cuda/src/caffe/layers/cudnn_conv_layer.o] Error 1de>
de >make: *** Waiting for unfinished jobs....de>

这是cuDNN版本不匹配造成的，参照：http://blog.csdn.net/yhl_leo/article/details/51371936 cuDNN当前最新版本是v5，近两三年的一些caffe工程，使用的版本不尽相同，其中以v2/v3版本的最为常见，所以使用的时候一定要搞清楚（当然如果作者没说，那就自己依次尝试吧）。
解决方法：

de >$ cd lib64/de>
de >$ sudo cp lib* /usr/local/cuda/lib64/de>
de >$ cd ../include/de>
de >$ sudo cp cudnn.h /usr/local/cuda/include/de>
de >$ cd /usr/local/cuda/lib64/de>
de >$ sudo rm -r libcudnn.so libcudnn.so.7.0de>
de >$ sudo ln -sf libcudnn.so.7.0.64 libcudnn.so.7.0de>
de >$ sudo ln -sf libcudnn.so.7.0 libcudnn.sode>
de >$ sudo ldconfigde>
但是由于之前已经有多个版本的cudnn在这个目录下，故存在冲突(1)如果使用cmake编译caffe，请修改一下文件，将cudnn的目录添加进lib和include路径中查看caffe -fast-cnn依赖项/fast-cnn/caffe-fast-cnn/cmake/Dependencies.cmake



(2)如果使用make编译caffe，则修改/fast-rcnn/caffe-fast-rcnn/Makefile


同时，注释掉已经存在的cudnn.h头文件，so不用改变，事实证明.h不共存，so可以共存。最后生成这几个文件（如make.log所示）：


5)下载Fast RCNN检测器
cd $FRCN_ROOT
./data/scripts/fetch_fast_rcnn_models.sh

你到目录下可以看到3个shell文件，分别是fetch_fast_rcnn_models.s h,fetch_imagenet_models.sh,fetch_selective_search_data.sh，第一是作者训练好的fast_rcnn模型，第二个是imagenet_model上预训练好的模型，第三个对应着的是作者基于Pascal VOC数据集提取的selective_search预选框。如果想要看一下fast rcnn的效果，可以直接加载Ross训练好的fast_rcnn模型，如果要自己训练的话，记得加载imagenet模型

de >#!/bin/bashde>
de >de>
de >DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )/../" && pwd )"de>
de >cd $DIRde>
de >de>
de >FILE=fast_rcnn_models.tgzde>
de >URL=http://www.cs.berkeley.edu/~rbg/fast-rcnn-data/$FILEde>
de >CHECKSUM=5f7dde9f5376e18c8e065338cc5df3f7de>
de >de>
de >if [ -f $FILE ]; thende>
de > echo "File already exists. Checking md5..."de>
de > os=`uname -s`de>
de > if [ "$os" = "Linux" ]; thende>
de > checksum=`md5sum $
c2e2
FILE | awk '{ print $1 }'`de>
de > elif [ "$os" = "Darwin" ]; thende>
de > checksum=`cat $FILE | md5`de>
de > fide>
de > if [ "$checksum" = "$CHECKSUM" ]; thende>
de > echo "Checksum is correct. No need to download."de>
de > exit 0de>
de > elsede>
de > echo "Checksum is incorrect. Need to download again."de>
de > fide>
de >fide>
de >de>
de >echo "Downloading Fast R-CNN demo models (0.96G)..."de>
de >de>
de >wget $URL -O $FILEde>
de >de>
de >echo "Unzipping..."de>
de >de>
de >tar zxvf $FILEde>
de >de>
de >echo "Done. Please run this command again to verify that checksum = $CHECKSUM."de>



6）Run the demo

PythonTo run the demo

cd $FRCN_ROOT
./tools/demo.py

The demo performs detection using a VGG16 network trained for detection on PASCAL VOC 2007. The object proposals are pre-computed in order to reduce installation requirements.Note: If the demo crashes Caffe because your GPU doesn't have enough memory, try running the demo with a small network, e.g., de style="box-sizing: border-box; font-size: 13.6px; padding: 0.2em 0px; margin-top: 0px; margin-bottom: 0px; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgba(0, 0, 0, 0.0392157);" >./tools/demo.py --net caffenetde> or with de style="box-sizing: border-box; font-size: 13.6px; padding: 0.2em 0px; margin-top: 0px; margin-bottom: 0px; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgba(0, 0, 0, 0.0392157);" >--net vgg_cnn_m_1024de>. Or run in CPU mode de style="box-sizing: border-box; font-size: 13.6px; padding: 0.2em 0px; margin-top: 0px; margin-bottom: 0px; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgba(0, 0, 0, 0.0392157);" >./tools/demo.py --cpude>. Typede style="box-sizing: border-box; font-size: 13.6px; padding: 0.2em 0px; margin-top: 0px; margin-bottom: 0px; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgba(0, 0, 0, 0.0392157);" >./tools/demo.py -hde> for usage.


将之前改变的cudnn.h改回原来的，保证原有的继续可以运行，而新的也可以运行。

来自为知笔记(Wiz)