TensorFlow学习——CIFAR-10(一)
2017-07-30 12:34
351 查看
本文学习参考视频:http://v.youku.com/v_show/id_XMjY4MjYzNzUwNA==.html?spm=a2h0k.8191407.0.0&from=s1.8-1-1.2
CIFAR-10官网: http://www.cs.toronto.edu/~kriz/cifar.html
TensorFlow处理二进制数据格式流程:
读取数据,数据不能一次读进内存中,需要生成相同大小的批次(batch),将batch feed进TensorFlow的计算图中,进而训练,验证;
![](https://oscdn.geek-share.com/Uploads/Images/Content/201911/10/957078bdf008a4741832cbed5d7d9911)
CIFAR-10数据集(多伦多大学,二进制格式存放)
Training Images: 50000彩色图片(32*32),共10类(飞机,汽车,鸟,猫,鹿,狗,蛙,马,船,卡车);
Testing Images: 10000图
每个二进制长度固定,一张图的二进制表述如下(图像类别字节+RGB图片像素字节):
![](https://oscdn.geek-share.com/Uploads/Images/Content/201911/10/9ea6cb2b7581f65067d5cf371b83f2a5)
CIFAR-10官网供三种数据格式下载(Python,MATLAB,二进制)
Python格式数据需要cPickle库支持load,但此种方式下载,会将数据一次性放入内存中,本次学习目的假设不能一次性将大量数据放入内存,因此选择二进制文件格式;
官网表述如下:
Python / Matlab versions
I will describe the layout of the Python version of the dataset. The layout of the Matlab version is identical.
The archive contains the files data_batch_1, data_batch_2,
..., data_batch_5, as well as test_batch.
Each of these files is a Python "pickled" object produced with cPickle.
Here is a python2 routine which will open such a file and return a dictionary:
And a python3 version:
..., data_batch_5.bin, as well as test_batch.bin.
Each of these files is formatted as follows:
In other words, the first byte is the label of the first image, which is a number in the range 0-9. The next 3072 bytes are the values of the pixels of the image.
The first 1024 bytes are the red channel values, the next 1024 the green, and the final 1024 the blue. The values are stored in row-major order, so the first 32 bytes are the red channel values of the first row of the image.
下载数据说明(二进制格式文件):
存放目录:D:\cat_VS_dog\TensorFlow-DataSets\CIFAR-10\cifar-10-batches-bin
![](https://oscdn.geek-share.com/Uploads/Images/Content/201911/10/87c8f0099ff8b2aba6c226e3c8f96ca3)
5个bin文件为训练集,将50000张图片压缩成5个bin文件(每个10000张);
测试图片集:test_batch.bin
batches.meta.txt:对训练集内容类别做了解释;
如何读数据及解码:
![](https://oscdn.geek-share.com/Uploads/Images/Content/201911/10/dfb2b6c9000fbea14c8164be99c71afd)
用tf.train.string_input_producer() 制作输入数据的队列;
用tf.FixedLengthRecordReader()读固定长度的数据格式(3073长度);
用tf.decode_raw()解码二进制格式文件;
解码后,将第一个元素提取给label,后面的3072长度reshape图片尺寸32*32;
返回上句提到的参数;
下一篇介绍代码实现
CIFAR-10官网: http://www.cs.toronto.edu/~kriz/cifar.html
TensorFlow处理二进制数据格式流程:
读取数据,数据不能一次读进内存中,需要生成相同大小的批次(batch),将batch feed进TensorFlow的计算图中,进而训练,验证;
CIFAR-10数据集(多伦多大学,二进制格式存放)
Training Images: 50000彩色图片(32*32),共10类(飞机,汽车,鸟,猫,鹿,狗,蛙,马,船,卡车);
Testing Images: 10000图
每个二进制长度固定,一张图的二进制表述如下(图像类别字节+RGB图片像素字节):
CIFAR-10官网供三种数据格式下载(Python,MATLAB,二进制)
Python格式数据需要cPickle库支持load,但此种方式下载,会将数据一次性放入内存中,本次学习目的假设不能一次性将大量数据放入内存,因此选择二进制文件格式;
官网表述如下:
Python / Matlab versions
I will describe the layout of the Python version of the dataset. The layout of the Matlab version is identical.
The archive contains the files data_batch_1, data_batch_2,
..., data_batch_5, as well as test_batch.
Each of these files is a Python "pickled" object produced with cPickle.
Here is a python2 routine which will open such a file and return a dictionary:
def unpickle(file): import cPickle with open(file, 'rb') as fo: dict = cPickle.load(fo) return dict
And a python3 version:
def unpickle(file): import pickle with open(file, 'rb') as fo: dict = pickle.load(fo, encoding='bytes') return dict
Binary version
The binary version contains the files data_batch_1.bin, data_batch_2.bin,..., data_batch_5.bin, as well as test_batch.bin.
Each of these files is formatted as follows:
<1 x label><3072 x pixel> ... <1 x label><3072 x pixel>
In other words, the first byte is the label of the first image, which is a number in the range 0-9. The next 3072 bytes are the values of the pixels of the image.
The first 1024 bytes are the red channel values, the next 1024 the green, and the final 1024 the blue. The values are stored in row-major order, so the first 32 bytes are the red channel values of the first row of the image.
下载数据说明(二进制格式文件):
存放目录:D:\cat_VS_dog\TensorFlow-DataSets\CIFAR-10\cifar-10-batches-bin
5个bin文件为训练集,将50000张图片压缩成5个bin文件(每个10000张);
测试图片集:test_batch.bin
batches.meta.txt:对训练集内容类别做了解释;
如何读数据及解码:
用tf.train.string_input_producer() 制作输入数据的队列;
用tf.FixedLengthRecordReader()读固定长度的数据格式(3073长度);
用tf.decode_raw()解码二进制格式文件;
解码后,将第一个元素提取给label,后面的3072长度reshape图片尺寸32*32;
返回上句提到的参数;
下一篇介绍代码实现
相关文章推荐
- TensorFlow学习——CIFAR-10(二)代码实现
- TensorFlow学习--卷积神经网络训练CIFAR-10数据集
- tensorflow学习笔记----二(CIFAR-10 模型 )
- Tensorflow学习笔记:CNN篇(3)——CIFAR-10数据集的CNN实现
- Tensorflow学习笔记:CNN篇(4)——CIFAR-10数据集LeNet实现(Keras版)
- Tensorflow深度学习之二十二:AlexNet的实现(CIFAR-10数据集)
- TensorFlow深度学习进阶教程:TensorFlow实现CIFAR-10数据集测试的卷积神经网络
- Tensorflow学习笔记:CNN篇(5)——CIFAR-10数据集VGG19实现(Keras版)
- TensorFlow学习笔记---CNN分类CIFAR-10数据集3
- tensorflow 学习之 cifar_10 模型定义 4000
- tensorflow 学习之 cifar_10 模型定义(补)
- TensorFlow学习——CIFAR-10(python实现数据可视化)
- TensorFlow-CNN CIFAR-10数据集 学习
- 深度学习小白——tensorflow(四)CIFAR-10实例
- tensorflow学习之cifar_10模型评估
- CIFAR-10 在Caffe上训练学习
- caffe读书笔记1 CIFAR-10在caffe上进行训练与学习
- 【神经网络与深度学习】基于Windows+Caffe的Minst和CIFAR—10训练过程说明
- Deep Learning-TensorFlow (4) CNN卷积神经网络_CIFAR-10进阶图像分类模型(上)
- caffe学习(一):CIFAR-10