Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
2017-07-06 10:51
579 查看
This repository contains pre-trained models and sampling code for the 3D Generative Adversarial Network (3D-GAN) presented at NIPS 2016.
http://3dgan.csail.mit.edu
3D-GAN which generates 3D objects from a probabilistic space by leveraging recent advances in volumetric convolutional networks and generative adversarial nets. The benefits of our model are
three-fold: first, the use of an adversarial criterion, instead of traditional heuristic
criteria, enables the generator to capture object structure implicitly and to synthesize high-quality 3D objects; second, the generator establishes a mapping from
a low-dimensional probabilistic space to the space of 3D objects, so that we can
sample objects without a reference image or CAD models, and explore the 3D
object manifold; third, the adversarial discriminator provides a powerful 3D shape
descriptor which, learned without supervision, has wide applications in 3D object
recognition.
recombining parts or pieces from a pre-defined repository to produce novel shapes; and for objects to
be realistic, there need to be fine details in the generated examples.
network to generate 3D objects. As shown in Figure 1, the generator consists of five volumetric fully
convolutional layers of kernel sizes 4 × 4 × 4 and strides 2, with batch normalization and ReLU
layers added in between and a Sigmoid layer at the end. The discriminator basically mirrors the
generator, except that it uses Leaky ReLU [Maas et al., 2013] instead of ReLU layers. There are no
pooling or linear layers in our network.
discriminator in every batch. However, the discriminator usually learns much faster than the generator,
possibly because generating objects in a 3D voxel space is more difficult than differentiating between
real and synthetic objects [Goodfellow et al., 2014, Radford et al., 2016]. It then becomes hard
for the generator to extract signals for improvement from a discriminator that is way ahead, as all
examples it generated would be correctly identified as synthetic with high confidence. Therefore,
to keep the training of both networks in pace, we employ an adaptive training strategy: for each
batch, the discriminator only gets updated if its accuracy in the last batch is not higher than 80%. We
observe this helps to stabilize the training and to produce better results. We set the learning rate of
G to 0:0025, D to 10−5, and use a batch size of 100. We use ADAM [Kingma and Ba, 2015] for
optimization, with β = 0:5.
实验部分主要是:1、可视化判别器;2、比较生成器,是否记住数据;3、插值,有一类物体线性插值到另一类物体,如从汽车逐渐变化到船
4、算术运算,即物体可以做bool运算,一个东西减去另一个东西得到其剩余部分,两个东西相加得到两个物体的和。
http://3dgan.csail.mit.edu
Prerequisites
论文介绍3D-GAN which generates 3D objects from a probabilistic space by leveraging recent advances in volumetric convolutional networks and generative adversarial nets. The benefits of our model are
three-fold: first, the use of an adversarial criterion, instead of traditional heuristic
criteria, enables the generator to capture object structure implicitly and to synthesize high-quality 3D objects; second, the generator establishes a mapping from
a low-dimensional probabilistic space to the space of 3D objects, so that we can
sample objects without a reference image or CAD models, and explore the 3D
object manifold; third, the adversarial discriminator provides a powerful 3D shape
descriptor which, learned without supervision, has wide applications in 3D object
recognition.
a generative model should be able to go beyond memorizing and
recombining parts or pieces from a pre-defined repository to produce novel shapes; and for objects to
be realistic, there need to be fine details in the generated examples.
随着large 3d cad数据集的引入,如ShapeNet,产生了一些有趣的想法:如基于体素的深度表示;不同于检索式,它是用深层表示来合成3d 物体的。即隐形编码2d图片,用于产生其深度特征表示,再利用其深度特征产生3d图像;本文方法将深度对抗网络和体元卷积网络融合,通过判别器来判断3d object is synthesized or real. 作者用高斯或者均值分布来生成3d object,并且利用无监督的方式。判别器和传统的3d物体判别其一样,可以用来判别一个输入是否为真实的3d物体。另外,作者实验了利用VAE,将2d图像作为输入,先产生其向量表示,再生成其对应的3d物体,其中当然有一定的限制,即输入图像的范围式一定的,不可任意输入。 近年来一些工作如:学习联合学习嵌入的3d shape和合成图像;学习判断3d object的判别式表达;利用循环网络进行3d重构;尝试生成3d图像;有些将图像用于3d像2d投射的过程中;大多数这些都是监督训练的,可以用来3d shape检索、分类、重构; Network structure Inspired by Radford et al. [2016], we design an all-convolutional neural
network to generate 3D objects. As shown in Figure 1, the generator consists of five volumetric fully
convolutional layers of kernel sizes 4 × 4 × 4 and strides 2, with batch normalization and ReLU
layers added in between and a Sigmoid layer at the end. The discriminator basically mirrors the
generator, except that it uses Leaky ReLU [Maas et al., 2013] instead of ReLU layers. There are no
pooling or linear layers in our network.
Training details A straightforward training procedure is to update both the generator and the
discriminator in every batch. However, the discriminator usually learns much faster than the generator,
possibly because generating objects in a 3D voxel space is more difficult than differentiating between
real and synthetic objects [Goodfellow et al., 2014, Radford et al., 2016]. It then becomes hard
for the generator to extract signals for improvement from a discriminator that is way ahead, as all
examples it generated would be correctly identified as synthetic with high confidence. Therefore,
to keep the training of both networks in pace, we employ an adaptive training strategy: for each
batch, the discriminator only gets updated if its accuracy in the last batch is not higher than 80%. We
observe this helps to stabilize the training and to produce better results. We set the learning rate of
G to 0:0025, D to 10−5, and use a batch size of 100. We use ADAM [Kingma and Ba, 2015] for
optimization, with β = 0:5.
实验部分主要是:1、可视化判别器;2、比较生成器,是否记住数据;3、插值,有一类物体线性插值到另一类物体,如从汽车逐渐变化到船
4、算术运算,即物体可以做bool运算,一个东西减去另一个东西得到其剩余部分,两个东西相加得到两个物体的和。
最后一个分是如何训练2d图像到3d物体的映射,如下图: 文章多处强调无监督学习,但没有提如何进行无监督训练。
相关文章推荐
- 论文阅读——Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
- Learning the parts of object by NMF
- 阅读笔记Learning Informative Point Classes for the Acquisition of Object Model Maps及Aligning Point......
- Projective Feature Learning for 3D Shapes with Multi-View Depth Images
- Dense Structural Learning for Infrared Object Tracking - DSLT: winner of VOT-TIR2017
- Distorting Object Shapes in Screen Space(屏幕空间扭曲)
- Learning part-based templates from large collections of 3D shapse CorrsTmplt Kim 代码调试
- 视觉学习中对象检测到的概率(2)(Probabilistic Visual Learning for Object Representation)
- More 3D Graphics (rgl) for Classification with Local Logistic Regression and Kernel Density Estimates (from The Elements of Statistical Learning)(转)
- 关于Myeclipse下的JVM启动异常的问题:Out Of Memery / Could not reserve enough space for object heap
- ant build Error occurred during initialization of VM Could not reserve enough space for object heap
- Error occurred during initialization of VM Could not reserve enough space for object heap Could not
- fisheye Error occurred during initialization of VM Could not reserve enough space for object heap 问题解决!
- Learning the parts of object by NMF
- [PoEAA] Learning Notes of Data Transfer Object
- Error occurred during initialization of VM Could not reserve enough space for object heap
- Identifying multiple/single values of metadata and recording into XML via .NET Client Object Model
- 视觉学习中对象检测到的概率(1)(Probabilistic Visual Learning for Object Representation)
- Error occurred during initialization of VM Could not reserve enough space for object heap
- android Error occurred during initialization of VM Could not reserve enough space for object heap Could not create the Java virtual machine.