keras learning rate

http://machinelearningmastery.com/using-learning-rate-schedules-deep-learning-models-python-keras/
https://stackoverflow.com/questions/39779710/setting-up-a-learningratescheduler-in-keras    (打印出每一个周期的学习率lr的代码)
https://github.com/fchollet/keras/issues/898
直接使用命令设置

import keras.backend as K
sgd = SGD(lr=0.1, decay=0, momentum=0.9, nesterov=True)
K.set_value(sgd.lr, 0.5 * K.get_value(sgd.lr))

或者编写回调类

class decay_lr(Callback):
'''
n_epoch = no. of epochs after decay should happen.
decay = decay value
'''
def __init__(self, n_epoch, decay):
super(decay_lr, self).__init__()
self.n_epoch=n_epoch
self.decay=decay

def on_epoch_begin(self, epoch, logs={}):
old_lr = self.model.optimizer.lr.get_value()
if epoch > 1 and epoch%self.n_epoch == 0 :
new_lr= self.decay*old_lr
k.set_value(self.model.optimizer.lr, new_lr)
else:
k.set_value(self.model.optimizer.lr, old_lr)

decaySchedule=decay_lr(10, 0.95)

With TF backend, I did this (for inception-V3)

from keras.callbacks import LearningRateScheduler
def scheduler(epoch):
if epoch%2==0 and epoch!=0:
lr = K.get_value(model.optimizer.lr)
K.set_value(model.optimizer.lr, lr*.9)
print("lr changed to {}".format(lr*.9))
return K.get_value(model.optimizer.lr)

lr_decay = LearningRateScheduler(scheduler)

model.fit_generator(train_gen, (nb_train_samples//batch_size)*batch_size,
nb_epoch=100, verbose=1,
validation_data=valid_gen,    nb_val_samples=val_size,
callbacks=[lr_decay])

以上都是以epoch为周期的，其实每一次minibatch就算一次update（例如model.train_on_batch()）， iteration的状态+1， 这就是学习率中decay作用时的iteration数值，并不等于epoch. 一个epoch有多少minibatch,就有多少iteration