工作记录：在自己的数据库上训练RCNN

参照RCNN官网，安装caffe0.999，安装RCNN，在VOC2007上试运行RCNN通过后，尝试在自己的数据库上训练。

1. DATABASE

注意数据库的结构：

模仿VOC2007，VOCdevkit/ 下：

VOCdevkit/
VOCdevkit/OPT2017/
VOCdevkit/OPT2017/Annotations/
VOCdevkit/OPT2017/ImageSets/
VOCdevkit/OPT2017/ImageSets/Main/
VOCdevkit/OPT2017/JPEGImages/
VOCdevkit/OPT2017/labels/
VOCdevkit/results/
VOCdevkit/results/OPT2017/
VOCdevkit/results/OPT2017/Main/
VOCdevkit/VOCcode/

具体每个文件夹下的内容参照标准VOC2007即可。

参照此标准，修改自己的数据库，使其符合标准。

建立数据集快捷方式：

（这里s表示软连接，f表示强制创建.）

ln -sf /home/user3/DATA/OPTIMAL_Z_FINAL/VOCdevkit /home/user3/CODE/rcnn/datasets/OPTdevkit2017

VOCcode准备：

rcnn/imdb/imdb_eval_voc.m:

line43

year = VOCopts.dataset(4:end);

line75

do_eval = (str2num(year) <= 2020) | ~strcmp(test_set, ‘test’);

看出，命名时应该按照xxx2007的格式命名（年份前三个字母），因此我将自己的数据库命名为OPT2017。

此外，我修改了line75中对年份的约束，如以上所示。

VOCdevkit/VOCcode/VOCinit.m:

在训练时要把VOCcode/VOCinit.m中的VOCopts.dataset即数据集名改为自己的数据集名字，VOCopts.classes即类别名改为自己的类别名字。

rcnn/imdb/准备：

imdb_from_voc.m:

修改部分：

30： cache_file = ['./imdb/cache/imdb_OPT_' year '_' image_set];
37： imdb.name = ['OPT_' year '_' image_set];

get_voc_opts.m:

–

imdb_eval_voc.m:

–

2. Fine-tuning a CNN for detection with Caffe ( Prepare )

图像文件

计算imdb，并保存为相应的文件：

>> imdb_train = imdb_from_voc('datasets/OPTdevkit2017', 'train', '2017');
>> imdb_val = imdb_from_voc('datasets/OPTdevkit2017', 'val', '2017');
>> imdb_test = imdb_from_voc('datasets/OPTdevkit2017', 'test', '2017');
>> imdb_trainval = imdb_from_voc('datasets/OPTdevkit2017', 'trainval', '2017');

selective_search 生成 proposal

然后生成proposal:

参考微博提供的github代码，只复制其

selective_search_rcnn.m

到

rcnn/selective_search/SelectiveSearchCodeIJCV/

目录下，并作部分修改（添加了计时功能以及显示目前进度的功能），我的修改结果：

function all_boxes = selective_search_rcnn(image_filenames, output_filename)

% Based on the demo.m file included in the Selective Search
% IJCV code, and on selective_search_boxes.m from R-CNN.

% Load dependencies and compile if needed.

addpath(fullfile(cd , 'selective_search/SelectiveSearchCodeIJCV/Dependencies'));

if(~exist('anigauss'))
mex Dependencies/anigaussm/anigauss_mex.c Dependencies/anigaussm/anigauss.c -output anigauss
end

if(~exist('mexCountWordsIndex'))
mex Dependencies/mexCountWordsIndex.cpp
end

if(~exist('mexFelzenSegmentIndex'))
mex Dependencies/FelzenSegment/mexFelzenSegmentIndex.cpp -output mexFelzenSegmentIndex;
end

% Configure
im_width = 500;

% Parameters. Note that this controls the number of hierarchical
% segmentations which are combined.
colorTypes = {'Hsv', 'Lab', 'RGI', 'H', 'Intensity'};

% Here you specify which similarity functions to use in merging
simFunctionHandles = {@SSSimColourTextureSizeFillOrig, ...
@SSSimTextureSizeFill, ...
@SSSimBoxFillOrig, ...
@SSSimSize};

% Thresholds for the Felzenszwalb and Huttenlocher segmentation algorithm.
% Note that by default, we set minSize = k, and sigma = 0.8.
% controls size of segments of initial segmentation.
ks = [50 100 150 300];
sigma = 0.8;

% After segmentation, filter out boxes which have a width/height smaller
% than minBoxWidth (default = 20 pixels).
minBoxWidth = 20;

% Comment the following three lines for the 'quality' version
colorTypes = colorTypes(1:2); % 'Fast' uses HSV and Lab
simFunctionHandles = simFunctionHandles(1:2); % Two different merging strategies
ks = ks(1:2);

% Process all images.
all_boxes = {};
for i=1:length(image_filenames)
tic

im = imread(image_filenames{i});
% Resize image to canonical dimensions since proposals aren't scale invariant.
scale = size(im, 2) / im_width;
im = imresize(im, [NaN im_width]);

idx = 1;
for j = 1:length(ks)
k = ks(j); % Segmentation threshold k
minSize = k; % We set minSize = k
for n = 1:length(colorTypes)
colorType = colorTypes{n};
[boxesT{idx} blobIndIm blobBoxes hierarchy priorityT{idx}] = ...
Image2HierarchicalGrouping(im, sigma, k, minSize, colorType, simFunctionHandles);
idx = idx + 1;
end
end
boxes = cat(1, boxesT{:}); % Con
e807
catenate boxes from all hierarchies
priority = cat(1, priorityT{:}); % Concatenate priorities

% Do pseudo random sorting as in paper
priority = priority .* rand(size(priority));
[priority sortIds] = sort(priority, 'ascend');
boxes = boxes(sortIds,:);

boxes = FilterBoxesWidth(boxes, minBoxWidth);
boxes = BoxRemoveDuplicates(boxes);

% Adjust boxes to cancel effect of canonical scaling.
boxes = (boxes - 1) * scale + 1;

boxes = FilterBoxesWidth(boxes, minBoxWidth);
boxes = BoxRemoveDuplicates(boxes);
all_boxes{i} = boxes;

time = toc;
[num2str(i) '/' num2str(length(image_filenames)) sprintf('\t\t%d s',time)]
end

if nargin > 1
all_boxes
save(output_filename, 'all_boxes', '-v7');
end

然后，在相同的目录下新建函数

selective_search_rcnn_util.m

：

function ss_boxes = selective_search_rcnn_util()

% 注意这里不要'trainval'
% selective_search具有随机性，
% 而我们要求'trainval'={'train','val'}，
% 因此不能用selective_search产生'trainval'
dbsets = {'train','val','test'};
dbname_pf1 = 'imdb_';
dbname_pf2 = 'OPT_2017_';
list={};
ss_boxes = {};

for i = 1:4
load([dbname_pf1 , dbname_pf2 , dbsets{i}]);
for j = 1:length(imdb.image_ids)
list{j} = imdb.image_at(j);
end
path = fullfile( cd , 'data/selective_search_data' , [dbname_pf2 , dbsets{i}] );
ss_boxes{i} = selective_search_rcnn(list,path);
end

函数写好之后，在命令行中运行：

>> ss_boxes = selective_search_rcnn_util()

加载保存的文件，并加载 voc2007 对应的文件：

>> train = load('data/selective_search_data/OPT_2017_train');
>> voc_07_train = load('data/selective_search_data/VOC_2007_train');

对比两者结构，发现生成的文件结构、变量的名称、变量的类型都不同：

train
- all_boxes <1X5401 cell>
- <Nx4 double> % 小数数值
voc_07_train
- boxes <1X2501 cell>
- <Nx4 double> % 整数数值
- images <1X2501 cell>
- <1X6 char>

因此需要写个

selective_search_util_further.m

进一步调整：

为了防止中间出错，先把中间文件打包存起来存在

/home/user3/CODE/rcnn/data/selective_search_data/pre_OPT_2017_trainvaltest/

以备不时之需。

function selective_search_util_further( )
%SELECTIVE_SEARCH_UTIL_FURTHER Summary of this function goes here
%   Detailed explanation goes here
roidb_train = load('/home/user3/CODE/rcnn/data/selective_search_data/pre_OPT_2017_trainvaltest/OPT_2017_train.mat');
roidb_val = load('/home/user3/CODE/rcnn/data/selective_search_data/pre_OPT_2017_trainvaltest/OPT_2017_val.mat');
roidb_test = load('/home/user3/CODE/rcnn/data/selective_search_data/pre_OPT_2017_trainvaltest/OPT_2017_test.mat');
imdb_train = load('/home/user3/CODE/rcnn/imdb/cache/imdb_OPT_2017_train');
imdb_val = load('/home/user3/CODE/rcnn/imdb/cache/imdb_OPT_2017_val');
imdb_test = load('/home/user3/CODE/rcnn/imdb/cache/imdb_OPT_2017_test');

images = imdb_train.imdb.image_ids';
boxes = roidb_train.all_boxes;
save('/home/user3/CODE/rcnn/data/selective_search_data/OPT_2017_train','images','boxes','-v7');
images = imdb_val.imdb.image_ids';
boxes = roidb_val.all_boxes;
save('/home/user3/CODE/rcnn/data/selective_search_data/OPT_2017_val','images','boxes','-v7');
images = imdb_test.imdb.image_ids';
boxes = roidb_test.all_boxes;
save('/home/user3/CODE/rcnn/data/selective_search_data/OPT_2017_test','images','boxes','-v7');

end

再写个.m生成对应的

OPT_2017_trainval

:

function selective_search_util_trainval( )
%SELECTIVE_SEARCH_UTIL_TRAINVAL Summary of this function goes here
%   Detailed explanation goes here
train = load('/home/user3/CODE/rcnn/data/selective_search_data/OPT_2017_train');
val =  load('/home/user3/CODE/rcnn/data/selective_search_data/OPT_2017_val');
imagesa = train.images;
imagesb = val.images;
boxesa = train.boxes;
boxesb = val.boxes;
images = sort( [imagesa,imagesb] );
for i = 1:length(images)
index = strmatch(images{i},imagesa);
if index
boxes{i} = boxesa{index};
else
index = strmatch(images{i},imagesb);
boxes{i} = boxesb{index};
end
end

save( 'data/selective_search_data/OPT_2017_trainval', 'images','boxes', '-v7' );

end

make windows for finetuning

加载imdb_train、imdb_val、imdb_test

imdb_train = imdb_from_voc('datasets/OPTdevkit2017', 'train', '2017');
imdb_val = imdb_from_voc('datasets/OPTdevkit2017', 'val', '2017');
imdb_test = imdb_from_voc('datasets/OPTdevkit2017', 'test', '2017');
imdb_trainval = imdb_from_voc('datasets/OPTdevkit2017', 'trainval', '2017');

make windows（得到 imdb/cache 目录下三个

roidb_OPT_*

）:

rcnn_make_window_file(imdb_train, 'external/caffe/examples/pascal-finetuning');
rcnn_make_window_file(imdb_val, 'external/caffe/examples/pascal-finetuning');
rcnn_make_window_file(imdb_test, 'external/caffe/examples/pascal-finetuning');

3. Run fine-tuning with Caffe

查看voc2012 的 prototxt

在线可视化

复制prototxt内容到该网站左侧栏，按shift + enter；其他方法

建立自己的配置文件

查看该目录下文件：

cd external/caffe/examples/pascal-finetuning/
ls

> results: pascal_finetune_solver.prototxt
> window_file_OPT_2017_test.txt
> pascal_finetune_train.prototxt
> window_file_OPT_2017_train.txt
> pascal_finetune_val.prototxt
> window_file_OPT_2017_val.txt

直接用VOC2012的配置文件的话会因为数据库名称不同而报错：

Check failed: infile.good() Failed to open window file window_file_voc_2012_train.txt

然而当前目录下却有

window_file_OPT_2017_*.txt

，因此，在voc2012 的 prototxt上修改，在自己的路径下写自己的训练结构文件：

/your/path/to/rcnn/finetuning/OPT_2017_prototxt

pascal_finetune_train.prototxt

pascal_finetune_val.prototxt

pascal_finetune_solver.prototxt

各个文件的修改部分如下：

pascal_finetune_train.prototxt

#line8:
source: "window_file_OPT_2017_train.txt"

pascal_finetune_val.prototxt

#line8:
source: "window_file_OPT_2017_val.txt"

pascal_finetune_solver.prototxt

#line14:
snapshot_prefix: "finetune_OPT_2017_train"

复制到工作路径下：

cp finetuning/OPT_2017_prototxt/pascal_finetune_* external/caffe/examples/pascal-finetuning/

准备imagenet_mean

以上工作都准备好之后，若直接开始训练，会报错：

找不到文件

data/ilsvrc12/imagenet_mean.binaryproto

因此要用imagenet工具生成相应的平均值文件。

而要达到此目的，需要生成的中间文件有：

# 由 create_imagenet.sh 生成
examples/imagenet/ilsvrc12_train_leveldb
examples/imagenet/ilsvrc12_val_leveldb
# 由 make_imagenet_mean.sh 生成
data/ilsvrc12/imagenet_mean.binaryproto

参考ImageNet的github官网，对应自己

rcnn/external/caffe

下的文件：

examples/imagenet/create_imagenet.sh
examples/imagenet/make_imagenet_mean.sh

或者自己生成太麻烦，直接参照Caffe训练ImageNet简介，直接下载imagenet_mean.binaryproto，解压后就有

imagenet_mean.binaryproto

，然后放到

/rcnn/external/caffe/data/ilsvrc12/

下即可。

finetune

最后，切换工作目录到：

external/caffe/examples/pascal-finetuning

，执行fine-tuning代码：

GLOG_logtostderr=1 ../../build/tools/finetune_net.bin  /home/user3/CODE/rcnn/pascal_finetune_solver.prototxt /home/user3/CODE/rcnn/data/caffe_nets/ilsvrc_2012_train_iter_310k 2>&1 | tee log.txt