kaggle | Digit recognizer with caffe
2016-06-19 20:29
609 查看
Maybe this competition is easy to complete but it might be a good chance to practice my Caffe skill.
https://www.kaggle.com/c/digit-recognizer
Firstly, maybe we need to know:
import
visuliza a digit
y[10]=8
Converting…
we get these lmdbs:
train_lmdb=301.7mb
see the details:
http://blog.csdn.net/bea_tree/article/details/51601197#t14
https://www.kaggle.com/c/digit-recognizer
1 Csv -> lmdb
http://www.cnblogs.com/dcsds1/p/5205669.htmlFirstly, maybe we need to know:
train.csv.shape= (42000, 785) test.csv.shape= (28000, 784) #train.csv's col_index #28*28=784 label pixel0 pixel1 ... pixel782 pixel783
import
import numpy as np import pandas as pd import lmdb import caffe import matplotlib.pyplot as plt from sklearn.cross_validation import train_test_split
visuliza a digit
path='/yourpath/' train_path=path+'train.csv' test_path=path+'test.csv' train_df=pd.read_csv(train_path)#42000 test_df=pd.read_csv(test_path)#28000 train_np=train_df.values y=train_np [:,0] X=train_np [:,1:] X=X.reshape((X.shape[0],1,28,28)) im1=X[10,0] print y[10] plt.rcParams['image.cmap']='gray' plt.imshow(im1) plt.show()
y[10]=8
Converting…
def covert_lmdb(X,y,path): m=X.shape[0] map_size=X.nbytes*10#donot worry , mapsize no harm # http://lmdb.readthedocs.io/en/release/#environment-class env=lmdb.open(path,map_size=map_size) # http://lmdb.readthedocs.io/en/release/#lmdb.Transaction with env.begin(write=True) as txn: for i in range(m): datum=caffe.proto.caffe_pb2.Datum() datum.channels=X.shape[1] datum.height=X.shape[2] datum.width=X.shape[3] datum.data=X[i].tostring()#tobeytes if np.version.version >1.9 datum.label=int(y[i]) str_id='{:08}'.format(i) txn.put(str_id.encode('ascii'),datum.SerializeToString()) train_lmdb_path=path+'train_lmdb' test_score_lmdb_path=path+'test_score_lmdb' covert_lmdb(X_train,y_train,train_lmdb_path) covert_lmdb(X_test,y_test,test_score_lmdb_path)
we get these lmdbs:
train_lmdb=301.7mb
2 Creating the net and solover
see the details:
http://blog.csdn.net/bea_tree/article/details/51601197#t14
def lenet(lmdb_path,batch_size): n=caffe.NetSpec() n.data,n.label=L.Data(batch_size=batch_size,backend=P.Data.LMDB,source=lmdb_path, transform_param=dict(scale=1./255),ntop=2) n.conv1=L.Convolution(n.data,kernel_size=5,num_output=20,weight_filler=dict(type='xavier')) n.pool1=L.Pooling(n.conv1,kernel_size=2,stride=2,pool=P.Pooling.MAX) n.conv2=L.Convolution(n.pool1,kernel_size=5,num_output=50,weight_filler=dict(type='xavier')) n.pool2 = L.Pooling(n.conv2, kernel_size=2, stride=2, pool=P.Pooling.MAX) n.fc1=L.InnerProduct(n.pool2,num_output=10,weight_filler=dict(type='xavier')) n.relu1 = L.ReLU(n.fc1, in_place=True) n.score=L.InnerProduct(n.relu1,num_output=10,weight_filler=dict(type='xavier')) n.loss=L.SoftmaxWithLoss(n.score,n.label) return n.to_proto() path='/media/beatree/file/note for project/work/conpetitions/kaggle/digit recognizer/data/' train_net_path=path+'mnist/lenet_train.prototxt' test_net_path=path+'mnist/lenet_test.prototxt' #with open(path+'mnist/lenet_train.prototxt','w') as f: # f.write(str(lenet(train_path,64))) #with open(path+'mnist/lenet_test.prototxt','w') as f: # f.write(str(lenet(test_path,64))) from caffe.proto import caffe_pb2 def solver(train_net_path,test_net_path): s=caffe_pb2.SolverParameter() s.train_net=train_net_path s.test_net.append(test_net_path) s.test_iter.append(66) s.test_interval=1000 s.base_lr=0.01 s.momentum=0.9 s.weight_decay=0.0005 s.lr_policy='inv' s.gamma=0.0001 s.power=0.75 s.display=100 s.max_iter=10000 s.snapshot=5000 s.snapshot_prefix=path+'mnist/lenet' s.solver_mode=caffe_pb2.SolverParameter.GPU return s #with open(path + 'mnist/solver.prototxt', 'w') as f: # f.write(str(solver(train_net_path, test_net_path)))
3 predict
path='/path/' test_path=path+'test.csv' prototxt_path=path+'lenet2.prototxt' caffemodel_path=path+'mnist/lenet_5_iter_10000.caffemodel' #caffemodel_path='/home/beatree/caffe-rc3/examples/mnist/model/lenet_iter_10000.caffemodel' caffe.set_device(0) caffe.set_mode_gpu() clf=caffe.Classifier(prototxt_path,caffemodel_path,image_dims=(28,28)) test=pd.read_csv(test_path) test_array=test.values n=np.shape(test_array)[0] a=[] for i in xrange(n): input=test_array[i,:]/255. input=input.reshape((1,28,28,1)) reslut=clf.predict(input,oversample=False).argmax() a.append(reslut) ImageId=[i+1 for i in range(n)] submit_pd=pd.DataFrame({'ImageId':ImageId,'Label':a}) submit_pd.to_csv(path+'model.csv',index=False)
4 result
this result is created by the model which we have done in caffe example.相关文章推荐
- CSS选择器
- web开发前端发展阶段
- CSS外边距合并
- jsTree搜索排序向上向下
- Caffe 实例笔记 3 Brewing Logistic Regression then Going Deeper
- caffe 实例笔记 2 LeNet详细解读及实现
- HTML常用标签
- HTML5 画布
- 用js写的一个手风琴效果
- 【剑指offer】O(1)时间删除链表结点
- javaScript系列:js中获取时间new Date()详细介绍
- C11 の multi-thread feature
- 找到给定数组中出现次数大于长度一半的数字,若没有,则返回0
- DHTML( radio的综合使用---模拟网上的调查问卷)
- markdownpad2注册及样式调整
- HTML总结
- jsp中对数据进行批量删除操作
- javascript
- 用CSS代码绘制三角形 纯CSS绘制三角形的代码
- CSS