caffe源码学习（六） 自定义层

经过前面对google protocol buffer、Blob、SyncedMemory 与 shared_ptr、layer、data layer的初步学习，已经能够仿照caffe中已有的层来写自定层了。所以接下来就在caffe深度学习框架下写自定义层。首先应该注意，不同版本的caffe可能会有一些区别，所以要根据官网指南来写自定义层。为了方便，把目前版本的指南（20160605）贴出来：

Developing new layers

1. Add a class declaration for your layer to

include/caffe/layers/your_layer.hpp

.

(1) Include an inline implementation of

type

overriding the method

virtual inline const char*  type() const { return "YourLayerName"; }

replacing

YourLayerName

with your layer’s name.

(2) Implement the

{*}Blobs()

methods to specify blob number requirements; see /caffe/include/caffe/layers.hpp to enforce strict top and bottom Blob counts using the inline

{*}Blobs()

methods.

(3) Omit the

*_gpu

declarations if you’ll only be implementing CPU code.

2. Implement your layer in

src/caffe/layers/your_layer.cpp

.

(1) (optional)

LayerSetUp

for one-time initialization: reading parameters, fixed-size allocations, etc.

(2)

Reshape

for computing the sizes of top blobs, allocating buffers, and any other work that depends on the shapes of bottom blobs

(3)

Forward_cpu

for the function your layer computes

(4)

Backward_cpu

for its gradient (Optional – a layer can be forward-only)

3. (Optional) Implement the GPU versions

Forward_gpu

and

Backward_gpu

in layers/your_layer.cu.

4. If needed, declare parameters in

proto/caffe.proto

, using (and then incrementing) the “next available layer-specific ID” declared in a comment above

message LayerParameter

.

5. Instantiate and register your layer in your cpp file with the macro provided in

layer_factory.hpp

. Assuming that you have a new layer

MyAwesomeLayer

, you can achieve it with the following command:

INSTANTIATE_CLASS(MyAwesomeLayer);
REGISTER_LAYER_CLASS(MyAwesome);

6. Note that you should put the registration code in your own cpp file, so your implementation of a layer is self-contained.

7. Optionally, you can also register a Creator if your layer has multiple engines. For an example on how to define a creator function and register it, see

GetConvolutionLayer

in

caffe/layer_factory.cpp

.

8. Write tests in

test/test_your_layer.cpp

. Use

test/test_gradient_check_util.hpp

to check that your Forward and Backward implementations are in numerical agreement.

Forward-Only Layers

If you want to write a layer that you will only ever include in a test net, you do not have to code the backward pass. For example, you might want a layer that measures performance metrics at test time that haven’t already been implemented. Doing this is very simple. You can write an inline implementation of

Backward_cpu

(or

Backward_gpu

) together with the definition of your layer in

include/caffe/your_layer.hpp

that looks like:

virtual void Backward_cpu(const vector<Blob<Dtype>*>& top, const vector<bool>& propagate_down, const vector<Blob<Dtype>*>& bottom) {
NOT_IMPLEMENTED;
}

The

NOT_IMPLEMENTED

macro (defined in

common.hpp

) throws an error log saying “Not implemented yet”. For examples, look at the accuracy layer (

accuracy_layer.hpp

) and threshold layer (

threshold_layer.hpp

) definitions.

官网指南中已经说的比较详细，下面自定义层主要用到1,2,3,4,5,6中提到的，简单的实现了

y = x^n

点对点的简单计算，其中n从prototxt文件中读取。7和8目前还没有实现，后面有需要的话再更新。然后通过matlab接口进行验证该层的计算是否正确。

还是按照老习惯，先给出源码，再做总结。

1.源码

mysquare.hpp(该代码是仿照caffe中已经实现的

absval_layer

和

dropout_layer

来写的，所以保留原有的一些内容。最开始要是实现

y = x^2

，接下来为了实现能够从prototxt中读取数据，实现

y = x^n

，n从prototxt中读取，所以命名为mysquare。)

#ifndef MYSQUARE_LAYER_HPP
#define MYSQUARE_LAYER_HPP

#include <vector>

#include "caffe/blob.hpp"
#include "caffe/layer.hpp"
#include "caffe/proto/caffe.pb.h"

#include "caffe/layers/neuron_layer.hpp"

namespace caffe {

/**
* @brief Computes @f$ y = x^2 @f$
*
* @param bottom input Blob vector (length 1)
*   -# @f$ (N \times C \times H \times W) @f$
*      the inputs @f$ x @f$
* @param top output Blob vector (length 1)
*   -# @f$ (N \times C \times H \times W) @f$
*      the computed outputs @f$ y = x^2 @f$
*/
template <typename Dtype>
class MySquareLayer : public NeuronLayer<Dtype> {
public:
explicit MySquareLayer(const LayerParameter& param)
: NeuronLayer<Dtype>(param) {}
virtual void LayerSetUp(const vector<Blob<Dtype>*>& bottom,
const vector<Blob<Dtype>*>& top);

virtual inline const char* type() const { return "MySquare"; }
virtual inline int ExactNumBottomBlobs() const { return 1; }
virtual inline int ExactNumTopBlobs() const { return 1; }

protected:
/// @copydoc MySquareLayer
virtual void Forward_cpu(const vector<Blob<Dtype>*>& bottom,
const vector<Blob<Dtype>*>& top);
virtual void Forward_gpu(const vector<Blob<Dtype>*>& bottom,
const vector<Blob<Dtype>*>& top);

/**
* @brief Computes the error gradient w.r.t. the absolute value inputs.
*
* @param top output Blob vector (length 1), providing the error gradient with
*      respect to the outputs
*   -# @f$ (N \times C \times H \times W) @f$
*      containing error gradients @f$ \frac{\partial E}{\partial y} @f$
*      with respect to computed outputs @f$ y @f$
* @param propagate_down see Layer::Backward.
* @param bottom input Blob vector (length 2)
*   -# @f$ (N \times C \times H \times W) @f$
*      the inputs @f$ x @f$; Backward fills their diff with
*      gradients @f$
*        \frac{\partial E}{\partial x} =
*            \mathrm{sign}(x) \frac{\partial E}{\partial y}
*      @f$ if propagate_down[0]
*/
virtual void Backward_cpu(const vector<Blob<Dtype>*>& top,
const vector<bool>& propagate_down, const vector<Blob<Dtype>*>& bottom);
virtual void Backward_gpu(const vector<Blob<Dtype>*>& top,
const vector<bool>& propagate_down, const vector<Blob<Dtype>*>& bottom);

float power_;
};

}  // namespace caffe

#endif // MYSQUARE_LAYER_HPP

mysquare_layer.cpp

#include <vector>

#include "caffe/layers/mysquare_layer.hpp"
#include "caffe/util/math_functions.hpp"

namespace caffe {

template <typename Dtype>
void MySquareLayer<Dtype>::LayerSetUp(const vector<Blob<Dtype>*>& bottom,
const vector<Blob<Dtype>*>& top) {
NeuronLayer<Dtype>::LayerSetUp(bottom, top);
CHECK_NE(top[0], bottom[0]) << this->type() << " Layer does not "
"allow in-place computation.";
power_ = this->layer_param_.mysquare_param().power();
}

template <typename Dtype>
void MySquareLayer<Dtype>::Forward_cpu(
const vector<Blob<Dtype>*>& bottom, const vector<Blob<Dtype>*>& top) {
const int count = top[0]->count();
Dtype* top_data = top[0]->mutable_cpu_data();
caffe_powx(count, bottom[0]->cpu_data(), Dtype(power_), top_data);
}

template <typename Dtype>
void MySquareLayer<Dtype>::Backward_cpu(const vector<Blob<Dtype>*>& top,
const vector<bool>& propagate_down, const vector<Blob<Dtype>*>& bottom) {
const int count = top[0]->count();
const Dtype* top_diff = top[0]->cpu_diff();
if (propagate_down[0]) {
const Dtype* bottom_data = bottom[0]->cpu_data();
Dtype* bottom_diff = bottom[0]->mutable_cpu_diff();
caffe_powx(count, bottom_data, Dtype(power_ - 1), bottom_diff);
caffe_scal(count, Dtype(power_), bottom_diff);
caffe_mul(count, bottom_diff, top_diff, bottom_diff);
}
}

#ifdef CPU_ONLY
STUB_GPU(MySquareLayer);
#endif

INSTANTIATE_CLASS(MySquareLayer);
REGISTER_LAYER_CLASS(MySquare);

}  // namespace caffe

mysquare_layer.cu

#include <vector>

#include "caffe/layers/mysquare_layer.hpp"
#include "caffe/util/math_functions.hpp"
#include <iostream>
using namespace std;

namespace caffe {

template <typename Dtype>
void MySquareLayer<Dtype>::Forward_gpu(
const vector<Blob<Dtype>*>& bottom, const vector<Blob<Dtype>*>& top) {
const int count = top[0]->count();
Dtype* top_data = top[0]->mutable_gpu_data();
caffe_gpu_powx(count, bottom[0]->gpu_data(), Dtype(power_), top_data);
}

template <typename Dtype>
void MySquareLayer<Dtype>::Backward_gpu(const vector<Blob<Dtype>*>& top,
const vector<bool>& propagate_down, const vector<Blob<Dtype>*>& bottom) {
const int count = top[0]->count();
const Dtype* top_diff = top[0]->gpu_diff();
if (propagate_down[0]) {
const Dtype* bottom_data = bottom[0]->gpu_data();
Dtype* bottom_diff = bottom[0]->mutable_gpu_diff();
const Dtype* bottom_data_w = bottom[0]->cpu_data();
const Dtype* bottom_diff_w = bottom[0]->cpu_diff();

cout << "bottom_data[0]: " << bottom_data_w[0] << endl;
cout << "bottom_diff[0]: " << bottom_diff_w[0] << endl;

caffe_gpu_powx(count, bottom_data, Dtype(power_ - 1), bottom_diff);

bottom_diff = bottom[0]->mutable_gpu_diff();
bottom_data_w = bottom[0]->cpu_data();
bottom_diff_w = bottom[0]->cpu_diff();
cout << "bottom_data[0]: " << bottom_data_w[0] << endl;
cout << "bottom_diff[0]: " << bottom_diff_w[0] << endl;

caffe_gpu_scal(count, Dtype(power_), bottom_diff);

bottom_diff = bottom[0]->mutable_gpu_diff();
bottom_data_w = bottom[0]->cpu_data();
bottom_diff_w = bottom[0]->cpu_diff();
cout << "bottom_data[0]: " << bottom_data_w[0] << endl;
cout << "bottom_diff[0]: " << bottom_diff_w[0] << endl;

caffe_gpu_mul(count, bottom_diff, top_diff, bottom_diff);

bottom_diff = bottom[0]->mutable_gpu_diff();
bottom_data_w = bottom[0]->cpu_data();
bottom_diff_w = bottom[0]->cpu_diff();
cout << "bottom_data[0]: " << bottom_data_w[0] << endl;
cout << "bottom_diff[0]: " << bottom_diff_w[0] << endl;
}
}

INSTANTIATE_LAYER_GPU_FUNCS(MySquareLayer);

}  // namespace caffe

caffe.proto

message LayerParameter{
...
++ optional MySquareParameter mysquare_param = 150;
...
}
...
++ message MySquareParameter {
++  optional float power = 1 [default = 2];
++ }
...
message V1LayerParameter{
...
++ MYSQUARE = 40;
...
}

deploy.prototxt

name: "CaffeNet"
layer {
name: "data"
type: "Input"
top: "data"
input_param { shape: { dim: 10 dim: 3 dim: 227 dim: 227 } }
}

layer {
name: "mysquare"
type: "MySquare"
bottom: "data"
top: "mysquare_out"
mysquare_param {
power: 3
}
}

test.m

% 在caffe_root/matlab下
im = imread('../examples/images/cat.jpg');
input_data = {prepare_image(im)};
input_data = {pp(im)};
m = 'test/deploy.prototxt'
caffe.set_mode_gpu();
caffe.set_device(1);
net = caffe.Net(m,'test')
out = net.forward(input_data);
ii = input_data{1,1};
oo = out{1,1};
% 简单观察一下计算是否正确
ii(1:10,1:10,1,1)
oo(1:10,1:10,1,1)
(-60.4030)^3

pp.m

% 在caffe_root/matlab下
% 下面是classification_demo.m中的一个函数。

function crops_data = pp(im)
% ------------------------------------------------------------------------
% caffe/matlab/+caffe/imagenet/ilsvrc_2012_mean.mat contains mean_data that
% is already in W x H x C with BGR channels
d = load('../+caffe/imagenet/ilsvrc_2012_mean.mat');
mean_data = d.mean_data;
IMAGE_DIM = 256;
CROPPED_DIM = 227;

% Convert an image returned by Matlab's imread to im_data in caffe's data
% format: W x H x C with BGR channels
im_data = im(:, :, [3, 2, 1]);  % permute channels from RGB to BGR
im_data = permute(im_data, [2, 1, 3]);  % flip width and height
im_data = single(im_data);  % convert from uint8 to single
im_data = imresize(im_data, [IMAGE_DIM IMAGE_DIM], 'bilinear');  % resize im_data
im_data = im_data - mean_data;  % subtract mean_data (already in W x H x C, BGR)

% oversample (4 corners, center, and their x-axis flips)
crops_data = zeros(CROPPED_DIM, CROPPED_DIM, 3, 10, 'single');
indices = [0 IMAGE_DIM-CROPPED_DIM] + 1;
n = 1;
for i = indices
for j = indices
crops_data(:, :, :, n) = im_data(i:i+CROPPED_DIM-1, j:j+CROPPED_DIM-1, :);
crops_data(:, :, :, n+5) = crops_data(end:-1:1, :, :, n);
n = n + 1;
end
end
center = floor(indices(2) / 2) + 1;
crops_data(:,:,:,5) = ...
im_data(center:center+CROPPED_DIM-1,center:center+CROPPED_DIM-1,:);
crops_data(:,:,:,10) = crops_data(end:-1:1, :, :, 5);

2.总结

郁闷啊，马上总结完了，不小心弄没了，以后有时间再更新吧。。。