the Overfitting in Neural Network
2008-04-09 10:19
771 查看
Related Definition:
1.Overfitting happens when a model works very well on its trainning set,but has poor performance on new data.
2.Overfitting is when the NN learns the specific details of the training set, instead of the general pattern found in all present and future data
A simple example is as follows. Given n points on 2-D space, do curve fitting. The optimal solution (i.e., no error) is a n+1 polynomial function. However, as it is easily know, such high order functions will be very irrgular. Intuitively, that will not be the real case.
overfitting 过拟合
a traditional problem of pattern recognition in SLT(statistical learning theory), the VC dim. of your nn(neural networks) is too high, so produce the phemenon, so in SVM(support vector machine) we fix the training error and reduce the VC dim. of nn.
There can be two causes:
1.Training for too long. Solution?
1.1 Test against a separate test set every so often.
1.2 Stop when the results on the test set start getting worse.
2.Too many hidden nodes
2.1 One node can model a linear function
2.2 More nodes can model higher-order functions, or more input patterns
2.3 Too many nodes model the training set too closely, preventing generalisation.
1.Overfitting happens when a model works very well on its trainning set,but has poor performance on new data.
2.Overfitting is when the NN learns the specific details of the training set, instead of the general pattern found in all present and future data
A simple example is as follows. Given n points on 2-D space, do curve fitting. The optimal solution (i.e., no error) is a n+1 polynomial function. However, as it is easily know, such high order functions will be very irrgular. Intuitively, that will not be the real case.
overfitting 过拟合
a traditional problem of pattern recognition in SLT(statistical learning theory), the VC dim. of your nn(neural networks) is too high, so produce the phemenon, so in SVM(support vector machine) we fix the training error and reduce the VC dim. of nn.
There can be two causes:
1.Training for too long. Solution?
1.1 Test against a separate test set every so often.
1.2 Stop when the results on the test set start getting worse.
2.Too many hidden nodes
2.1 One node can model a linear function
2.2 More nodes can model higher-order functions, or more input patterns
2.3 Too many nodes model the training set too closely, preventing generalisation.
相关文章推荐
- 知识蒸馏(Distillation)相关论文阅读(1)——Distilling the Knowledge in a Neural Network(以及代码复现)
- Distilling the Knowledge in a Neural Network 阅读笔记
- 蒸馏神经网络(Distill the Knowledge in a Neural Network) 论文笔记
- 论文:Distilling the Knowledge in a Neural Network
- What is the role of the activation function in a neural network?
- 「Deep Learning」Dropout - Drop out the Units in a Neural Network
- 深度学习论文笔记--Recover Canonical-View Faces in the Wild with Deep Neural Network
- 蒸馏神经网络(Distill the Knowledge in a Neural Network)
- Distilling the Knowledge in a Neural Network[论文阅读笔记]
- Techniques for preventing overfitting in training Deep Neural Networks
- 【论文笔记】Recover Canonical-View Faces in the Wild with Deep Neural Network
- Get the IP address of a network interface in C using SIOCGIFADDR
- 《Named Entity Recognition in Chinese Clinical Text Using Deep Neural Network》——笔记
- CNN数玉米穗--TasselNet: Counting maize tassels in the wild via local counts regression network
- Deep Learning: Assuming a deep neural network is properly regulated, can adding more layers actually make the performance degrade?
- QUEUEING IN THE LINUX NETWORK STACK
- The Role of DTV Network Interfaces in Media Controller Support for DVB
- The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software by Herb Sutter
- The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software 转贴
- E: GPG error: http://mirrors.oschina.net trusty-backports InRelease: Clearsigned file isn't valid, got 'NODATA' (does the network require authentication?)