How does deep residual learning work?
2017-02-06 16:42
681 查看
Deep Residual Learning network is a very intriguing network that was developed by researchers from Microsoft Research. The results are quite impressive in that it received first place in ILSVRC 2015 image classification. The network that they used had 152 layers,
an impressive 8 times deeper than a comparable VGG network. This is a snapshot from the paper: http://arxiv.org/pdf/1512.03385v... comparing
their network with a similarly constructed VGG Convolution Network:
The claim however by Jurgen Schmidhuber is that it is the same thing as an LSTM without gates. (see: Microsoft
Wins ImageNet 2015 through Feedforward LSTM without Gates ). Which does seem accurate if you take a look at what an LSTM node looks like:
In other words the inputs of a lower layer is made available to a node in a higher layer. The difference of course is that the Microsoft Residual Network when applied to image classification tasks employs convolution processing layers in its construction. Schmidhuber's
research group has published results of "Highway Networks": http://arxiv.org/pdf/1507.06228v... with
depths up to 100 layers.
However despite the similarities between LSTM and Highway Networks with Residual Network, the results are quite impressive in that it shows state-of-the-art results for a very deep neural network of 152 layers. A recent paper from Weizmann Institute of Science http://arxiv.org/pdf/1512.03965.... has
a mathematical proof that reveals the utility of having deeper networks than that of wider networks. The implication of these three results are that future Deep Learning progress will lead to the development of even deeper networks.
Google's GoogleNet has 22 layers, this was published in late 2014. Two generations later, Google mentioned its Inception 7 network that had 50+ layers.
In all the Residual, Highway and Inception networks, you will notice that the same inputs do travel through paths and different number of layers.
The trend is pretty clear. Not only are Deeper Neural Networks more accurate, they in addition require less weights.
Update: Two recent papers have shown (1) Residual Nets being equivalent to RNN and (2) Residuals Nets acting more like ensembles across several layers.
原文地址: https://www.quora.com/How-does-deep-residual-learning-work
an impressive 8 times deeper than a comparable VGG network. This is a snapshot from the paper: http://arxiv.org/pdf/1512.03385v... comparing
their network with a similarly constructed VGG Convolution Network:
The claim however by Jurgen Schmidhuber is that it is the same thing as an LSTM without gates. (see: Microsoft
Wins ImageNet 2015 through Feedforward LSTM without Gates ). Which does seem accurate if you take a look at what an LSTM node looks like:
In other words the inputs of a lower layer is made available to a node in a higher layer. The difference of course is that the Microsoft Residual Network when applied to image classification tasks employs convolution processing layers in its construction. Schmidhuber's
research group has published results of "Highway Networks": http://arxiv.org/pdf/1507.06228v... with
depths up to 100 layers.
However despite the similarities between LSTM and Highway Networks with Residual Network, the results are quite impressive in that it shows state-of-the-art results for a very deep neural network of 152 layers. A recent paper from Weizmann Institute of Science http://arxiv.org/pdf/1512.03965.... has
a mathematical proof that reveals the utility of having deeper networks than that of wider networks. The implication of these three results are that future Deep Learning progress will lead to the development of even deeper networks.
Google's GoogleNet has 22 layers, this was published in late 2014. Two generations later, Google mentioned its Inception 7 network that had 50+ layers.
In all the Residual, Highway and Inception networks, you will notice that the same inputs do travel through paths and different number of layers.
The trend is pretty clear. Not only are Deeper Neural Networks more accurate, they in addition require less weights.
Update: Two recent papers have shown (1) Residual Nets being equivalent to RNN and (2) Residuals Nets acting more like ensembles across several layers.
原文地址: https://www.quora.com/How-does-deep-residual-learning-work
相关文章推荐
- Why does deep learning work?
- How Does #DeepDream Work?
- When Does Deep Learning Work Better Than SVMs or Random Forests?
- How does netstat work
- 【转载】卷积神经网络CNN经典模型整理(AlexNet,GoogleNet,VGG,Deep Residual Learning)
- How does anti-virus software work?(转载)
- how to improve deep learning performance
- how does startup code work
- Oracle Ksplice如何工作?How does Ksplice work?
- 《Machine Learning is Fun Part 6: How to do Speech Recognition with Deep Learning》译文
- Deep Residual Learning for Image Recognition 笔记
- How does ElementName Binding work?
- Deep Residual Learning for Image Recognition(残差网络)
- How does LGWR/LNS process work in 10gR2 using LGWR ASYNC (文档 ID 1057898.1)
- 论文-Deep Residual Learning for Image Recognition
- ResnetCrowd A Residual Deep Learning Architecture for Crowd Counting,Violent Behaviour Detection
- How Does Proxy ARP Work?
- How does it work in C#? - Part 3 (C# LINQ in detail)
- 图像识别的深度残差学习Deep Residual Learning for Image Recognition
- 卷积神经网络CNN经典模型整理Lenet,Alexnet,Googlenet,VGG,Deep Residual Learning