caffe Sigmoid cross entropy loss 交叉熵损失函数
2017-12-05 08:36
1966 查看
Sigmoid 交叉熵损失函数(Sigmoid Cross Entropy Loss)
study(3) 关于激活函数以及loss function" title="caffe study(3) 关于激活函数以及loss function" style="box-sizing: border-box; border: 0px; vertical-align: middle; outline: 0px; max-width: 100%; margin: 0px; padding: 0px; list-style: none;" />
官方: loss
,输入:形状:
得分
, 这个层使用 sigmoid 函数
映射到概率分布
形状:
标签
输出:形状:
计算公式:
应用场景:
预测目标概率分布Parameters
bottom | input Blob vector (length 2) the scores , which this layer maps to probability predictions using the sigmoid function (see SigmoidLayer). the targets |
top | output Blob vector (length 1) the computed cross-entropy loss: |
top | output Blob vector (length 1), providing the error gradient with respect to the outputs This Blob's diff will simply contain the loss_weight* , as is the coefficient of this layer's output in the overall Netloss ; hence . (*Assuming that this top Blob is not used as a bottom (input) by any other layer of the Net.) |
propagate_down | see Layer::Backward. propagate_down[1] must be false as gradient computation with respect to the targets is not implemented. |
bottom | input Blob vector (length 2) the predictions ; Backward computes diff the labels – ignored as we can't compute their error gradients |
#include <cfloat>
#include <vector>
#include "caffe/layer.hpp"
#include "caffe/util/math_functions.hpp"
#include "caffe/vision_layers.hpp"
// Computes the cross-entropy (logistic) loss ,it is often used for predicting targets interpreted
// as probabilities. Detailed reference to the official document:
// http://caffe.berkeleyvision.org/doxygen/classcaffe_1_1SigmoidCrossEntropyLossLayer.html
namespace caffe {
template <typename Dtype>
void SigmoidCrossEntropyLossLayer<Dtype>::LayerSetUp(
const vector<Blob<Dtype>*>& bottom, const vector<Blob<Dtype>*>& top) {
LossLayer<Dtype>::LayerSetUp(bottom, top);
sigmoid_bottom_vec_.clear();
sigmoid_bottom_vec_.push_back(bottom[0]);
sigmoid_top_vec_.clear();
sigmoid_top_vec_.push_back(sigmoid_output_.get());
sigmoid_layer_->SetUp(sigmoid_bottom_vec_, sigmoid_top_vec_);
}
template <typename Dtype>
void SigmoidCrossEntropyLossLayer<Dtype>::Reshape(
const vector<Blob<Dtype>*>& bottom, const vector<Blob<Dtype>*>& top) {
LossLayer<Dtype>::Reshape(bottom, top);
CHECK_EQ(bottom[0]->count(), bottom[1]->count()) <<
"SIGMOID_CROSS_ENTROPY_LOSS layer inputs must have the same count.";
sigmoid_layer_->Reshape(sigmoid_bottom_vec_, sigmoid_top_vec_);
}
template <typename Dtype>
void SigmoidCrossEntropyLossLayer<Dtype>::Forward_cpu(
const vector<Blob<Dtype>*>& bottom, const vector<Blob<Dtype>*>& top) {
// The forward pass computes the sigmoid outputs.
sigmoid_bottom_vec_[0] = bottom[0];
sigmoid_layer_->Forward(sigmoid_bottom_vec_, sigmoid_top_vec_);
// Compute the loss (negative log likelihood)
const int count = bottom[0]->count();
const int num = bottom[0]->num();
// Stable version of loss computation from input data
const Dtype* input_data = bottom[0]->cpu_data();
const Dtype* target = bottom[1]->cpu_data();
Dtype loss = 0;
for (int i = 0; i < count; ++i) {
loss -= input_data[i] * (target[i] - (input_data[i] >= 0)) -
log(1 + exp(input_data[i] - 2 * input_data[i] * (input_data[i] >= 0)));
}
top[0]->mutable_cpu_data()[0] = loss / num;
}
template <typename Dtype>
void SigmoidCrossEntropyLossLayer<Dtype>::Backward_cpu(
const vector<Blob<Dtype>*>& top, const vector<bool>& propagate_down,
const vector<Blob<Dtype>*>& bottom) {
if (propagate_down[1]) {
LOG(FATAL) << this->type()
<< " Layer cannot backpropagate to label inputs.";
}
if (propagate_down[0]) {
// First, compute the diff
const int count = bottom[0]->count();
const int num = bottom[0]->num();
const Dtype* sigmoid_output_data = sigmoid_output_->cpu_data();
const Dtype* target = bottom[1]->cpu_data();
Dtype* bottom_diff = bottom[0]->mutable_cpu_diff();
caffe_sub(count, sigmoid_output_data, target, bottom_diff);
// Scale down gradient
const Dtype loss_weight = top[0]->cpu_diff()[0];
caffe_scal(count, loss_weight / num, bottom_diff);
}
}
#ifdef CPU_ONLY
STUB_GPU_BACKWARD(SigmoidCrossEntropyLossLayer, Backward);
#endif
INSTANTIATE_CLASS(SigmoidCrossEntropyLossLayer);
REGISTER_LAYER_CLASS(SigmoidCrossEntropyLoss);
} // namespace caffe
相关文章推荐
- caffe Sigmoid cross entropy loss 交叉熵损失函数
- 学习Caffe(五)浅析softmax cross entropy loss与sigmoid cross entropy loss
- caffe 加权交叉熵损失函数层(weighted sigmoid_cross_entropy_loss_layer)添加方法
- Caffe softmax_cross_entropy_loss_layer.cpp
- Caffe Loss 层 - SigmoidCrossEntropyLoss 推导与Python实现
- What are the differences between maximum likelihood and minimize cross entropy loss function?
- sigmoid_cross_entropy_with_logits and weighted_cross_entropy_with_logits
- Multinomial Logistic Loss and Cross Entropy Loss are the same
- [tensorflow损失函数系列]sigmoid_cross_entropy_with_logits
- SVM hinge loss / SoftMax cross entropy loss
- 深度学习--手写数字识别<二>--交叉熵损失函数(cross entropy cost function)
- 交叉熵损失(Cross Entropy Loss)计算过程
- ValueError: Only call `sigmoid_cross_entropy_with_logits` with named arguments (labels=..., logits=.
- sigmoid cross entorpy loss
- tensorflow函数--sigmoid_cross_entropy_with_logits
- 人脸识别:损失函数之softmax loss和cross entropy Loss
- 关于Logistic Regression、Softmax Loss和Cross-entropy的随笔
- Only call `sigmoid_cross_entropy_with_logits` with named arguments解决
- How to choose cross-entropy loss in tensorflow?
- caffe SigmoidCrossEntropyLossLayer 理论代码学习