Where to apply dropout in recurrent neural networks for handwriting recognition?
2016-12-08 11:58
393 查看
Bluche T, Kermorvant C, Louradour J. Where to apply dropout in recurrent neural networks for handwriting recognition?[C]// 2015 13th International Conference
on Document Analysis and Recognition (ICDAR). IEEE Computer Society, 2015:681-685.
paper to read:Improving neural networks by preventing co-adaption of feature detectors
recurrent neural network regularization
investigation of recurrent neural networks architectures and learning methods for spoken language understanding
Long short-term memory
dropout用在lstm前效果优于中间，优于后面
各层全加dropout不如在最优的位置添加dropout
加入dropout后，权重更加sharper，
LSTM层之后的dropout可以促使不同unit发掘更多的信息，减少相邻unit之间的依赖
LSTM层之前的dropout可以减少相邻像素点之间的依赖
on Document Analysis and Recognition (ICDAR). IEEE Computer Society, 2015:681-685.
paper to read:Improving neural networks by preventing co-adaption of feature detectors
recurrent neural network regularization
investigation of recurrent neural networks architectures and learning methods for spoken language understanding
Long short-term memory
dropout用在lstm前效果优于中间，优于后面
各层全加dropout不如在最优的位置添加dropout
加入dropout后，权重更加sharper，
LSTM层之后的dropout可以促使不同unit发掘更多的信息，减少相邻unit之间的依赖
LSTM层之前的dropout可以减少相邻像素点之间的依赖
内容来自用户分享和网络整理，不保证内容的准确性，如有侵权内容，可联系管理员处理 
相关文章推荐
- Dropout:A Simple Way to Prevent Neural Networks from Overfitting
- (Mike Lynch)Application of linear weight neural networks to recognition of hand print characters
- 论文笔记 Ensemble of Deep Convolutional Neural Networks for Learning to Detect Retinal Vessels in Fundus
- 译：《Dropout: A Simple Way to Prevent Neural Networks from Overfitting》
- an empirical study of learning rates in deep neural networks for speech recognition 总结
- Dropout: A Simple Way to Prevent Neural Networks from Over tting 论文阅读
- [文献阅读]dropout - a simple way to prevent neural networks from overfitting
- READING NOTE: Spatially Supervised Recurrent Convolutional Neural Networks for Visual Object Trackin
- Dropout: A Simple Way to Prevent Neural Networks from Overtting(泛读)
- Convolutional Neural Networks for Visual Recognition 5
- Techniques for preventing overfitting in training Deep Neural Networks
- Convolutional Neural Networks for Visual Recognition 6
- Convolutional Neural Networks for Visual Recognition 3
- Two-Stream Convolutional Networks for Action Recognition in Video
- Unable to load native-hadoop library for your platform... using builtin-java classes where applicab
- HADOOP:WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable终于解决了
- Unable to load native-hadoop library for your platform... using builtin-java classes where applicabl
- Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
- Convolutional Neural Networks for Visual Recognition 7
- Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition