神经网络的两种正则化方法

神经网络的 三种正则化方法：

1，和线性回归正则化类似，

,因为是多层结构，所以相应地正则化也为多层，假设一个两层的神经网络的正则化为：



2，第二种正则化的方法：Early Stopping，In machine learning, [b]early stopping is a form of regularization used when a machine learningmodel (such as a neural network) is trained by on-line gradient descent. In early stopping, thetraining set is split into a new training set and a validation set. Gradient descent is applied to the new training set. After each sweep through the new training set, the network is evaluated on the validation set. When the performance with the validation test stops improving, the algorithm halts. The network with the best performance on the validation set is then used for actual testing, with a separate set of data (the validation set is used in learning to decide when to stop).（wikipedia上的 ），其实就是使用一个验证集来确定迭代时间：当验证集的err开始上升时即表示开始存在过拟合，在这个点stop被认为是比较好的。[/b]