理解AC_GAN源码中遇到的一些函数（–小白笔记）

代码地址：链接

作者对应的叙述文章

[目录]

tfconstant

tfrandom_normal tftruncated_normal tfrandom_uniform

tfconcat

tfcast

tfmultinomial

tfsqueeze

tfrandom_normal tftruncated_normal tfrandom_uniform

tfone_hot

slimfully_connected

slimconv2d_transpose

slimflatteninputsoutputs_collectionsNonescopeNone

tftrainAdamOptimizer

tf.constant

tf.constant(value,dtype=None,shape=None,name=’Const’)

创建一个常量tensor，按照给出value来赋值，可以用shape来指定其形状。value可以是一个数，也可以是一个list。

如果是一个数，那么这个常亮中所有值的按该数来赋值。

如果是list,那么len(value)一定要小于等于shape展开后的长度。赋值时，先将value中的值逐个存入。不够的部分，则全部存入value的最后一个值。

a = tf.constant(2,shape=[2])
b = tf.constant(2,shape=[2,2])
c = tf.constant([1,2,3],shape=[6])
d = tf.constant([1,2,3],shape=[3,2])

sess = tf.InteractiveSession()
print(sess.run(a))
#[2 2]
print(sess.run(b))
#[[2 2]
# [2 2]]
print(sess.run(c))
#[1 2 3 3 3 3]
print(sess.run(d))
#[[1 2]
# [3 3]
# [3 3]]

tf.random_normal | tf.truncated_normal | tf.random_uniform

tf.random_normal(shape,mean=0.0,stddev=1.0,dtype=tf.float32,seed=None,name=None)

tf.truncated_normal(shape, mean=0.0, stddev=1.0, dtype=tf.float32, seed=None, name=None)

tf.random_uniform(shape,minval=0,maxval=None,dtype=tf.float32,seed=None,name=None)

这几个都是用于生成随机数tensor的。尺寸是shape

random_normal: 正太分布随机数，均值mean,标准差stddev

truncated_normal:截断正态分布随机数，均值mean,标准差stddev,不过只保留[mean-2*stddev,mean+2*stddev]范围内的随机数

random_uniform:均匀分布随机数，范围为[minval,maxval]

sess = tf.InteractiveSession()
x = tf.random_normal(shape=[1,5],mean=0.0,stddev=1.0,dtype=tf.float32,seed=None,name=None)
print(sess.run(x))
#===>[[-0.36128798  0.58550537 -0.88363433 -0.2677258   1.05080092]]

tf.concat

tf.concat(concat_dim, values, name=’concat’)

tf.concat是连接两个矩阵的操作,除去name参数用以指定该操作的name，与方法有关的一共两个参数：

第一个参数concat_dim：必须是一个数，表明在哪一维上连接.

如果concat_dim是0，那么在某一个shape的第一个维度上连，对应到实际，就是叠放到列上.

t1 = [[1, 2, 3], [4, 5, 6]]
t2 = [[7, 8, 9], [10, 11, 12]]
tf.concat(0, [t1, t2]) == >
d9f3
[[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12]]

如果concat_dim是1，那么在某一个shape的第二个维度上连

t1 = [[1, 2, 3], [4, 5, 6]]
t2 = [[7, 8, 9], [10, 11, 12]]
tf.concat(1, [t1, t2]) ==> [[1, 2, 3, 7, 8, 9], [4, 5, 6, 10, 11, 12]]

如果有更高维，最后连接的依然是指定那个维：

values[i].shape = [D0, D1, … Dconcat_dim(i), …Dn]连接后就是：[D0, D1, … Rconcat_dim, …Dn]

第二个参数values：就是两个或者一组待连接的tensor了

这里要注意的是：如果是两个向量，它们是无法调用tf.concat(1, [t1, t2]) 来连接的，因为它们对应的shape只有一个维度，当然不能在第二维上连了，虽然实际中两个向量可以在行上连，但是放在程序里是会报错的

如果要连，必须要调用tf.expand_dims来扩维：

t1=tf.constant([1,2,3])
t2=tf.constant([4,5,6])
#concated = tf.concat(1, [t1,t2])这样会报错
t1=tf.expand_dims(tf.constant([1,2,3]),1)
t2=tf.expand_dims(tf.constant([4,5,6]),1)
concated = tf.concat(1, [t1,t2])#这样就是正确的

tf.cast

cast(x, dtype, name=None)

将x的数据格式转化成dtype.例如，原来x的数据格式是bool，那么将其转化成float以后，就能够将其转化成0和1的序列。反之也可以

a = tf.Variable([1,0,0,1,1])
b = tf.cast(a,dtype=tf.bool)
sess = tf.InteractiveSession()
sess.run(tf.initialize_all_variables())
print(sess.run(b))
#[ True False False  True  True]

tf.multinomial

tf.multinomial(logits, num_samples, seed=None, name=None)

Args:

logits: 2-D Tensor with shape [batch_size, num_classes]. Each slice [i, :] represents the unnormalized log probabilities for all classes.

num_samples: 0-D. Number of independent samples to draw for each row slice.

seed: A Python integer. Used to create a random seed for the distribution. See tf.set_random_seed for behavior.

name: Optional name for the operation.

tf.squeeze()

squeeze(input, axis=None, name=None, squeeze_dims=None)

squeeze函数用于删除张量中的单一维(Removes dimensions of size 1 from the shape of a tensor）

Args:

–input: A

Tensor

. The

input

to squeeze.

–axis: An optional list of

ints

. Defaults to

[]

.If specified, only squeezes the dimensions listed. The dimension index starts at 0. It is an error to squeeze a dimension that is not 1.

– name: A name for the operation (optional).

– squeeze_dims: Deprecated keyword argument that is now axis.

Returns:

A

Tensor

. Has the same type as

input

.

Contains the same data as

input

, but has one or more dimensions of size 1 removed.

prettyprint

# 't' is a tensor of shape [1, 2, 1, 3, 1, 1]

shape(squeeze(t)) ==> [2, 3]

prettyprint

# 't' is a tensor of shape [1, 2, 1, 3, 1, 1]

shape(squeeze(t, [2, 4])) ==> [1, 2, 3, 1]

tf.random_normal | tf.truncated_normal | tf.random_uniform

tf.random_normal(shape,mean=0.0,stddev=1.0,dtype=tf.float32,seed=None,name=None)

tf.truncated_normal(shape, mean=0.0, stddev=1.0, dtype=tf.float32, seed=None, name=None)

tf.random_uniform(shape,minval=0,maxval=None,dtype=tf.float32,seed=None,name=None)

这几个都是用于生成随机数tensor的。尺寸是shape

random_normal: 正太分布随机数，均值mean,标准差stddev

truncated_normal:截断正态分布随机数，均值mean,标准差stddev,不过只保留[mean-2*stddev,mean+2*stddev]范围内的随机数

random_uniform:均匀分布随机数，范围为[minval,maxval]

tf.one_hot

a = tf.one_hot(indices, depth=10, on_value=None, off_value=None, axis=None, dtype=None, name=None)

Args:

indices: A Tensor of indices.

depth: A scalar defining the depth of the one hot dimension.

on_value: A scalar defining the value to fill in output when indices[j] = i. (default: 1)

off_value: A scalar defining the value to fill in output when indices[j] != i. (default: 0)

axis: The axis to fill (default: -1, a new inner-most axis).

dtype: The data type of the output tensor.

Returns:

output: The one-hot tensor.

examples:

indices = [0, 2, -1, 1]
depth = 3
on_value = 5.0
off_value = 0.0
axis = -1
```
输出是[4 *3]
```
output =
[5.0 0.0 0.0]  // one_hot(0)
[0.0 0.0 5.0]  // one_hot(2)
[0.0 0.0 0.0]  // one_hot(-1)

例2:

indices = [[0, 2], [1, -1]]

depth = 3

on_value = 1.0

off_value = 0.0

axis = -1

输出是[2*2*3]

output =

[

[1.0, 0.0, 0.0]  // one_hot(0)

[0.0, 0.0, 1.0]  // one_hot(2)

][

[0.0, 1.0, 0.0]  // one_hot(1)

[0.0, 0.0, 0.0]  // one_hot(-1)

]

slim.fully_connected()

fully_connected(inputs,

num_outputs,

activation_fn=nn.relu,

normalizer_fn=None,

normalizer_params=None,

weights_initializer=initializers.xavier_initializer(),

weights_regularizer=None,

biases_initializer=init_ops.zeros_initializer(),

biases_regularizer=None,

reuse=None,

variables_collections=None,

outputs_collections=None,

trainable=True,

scope=None)

函数产生一个名为weights的变量，代表一个全链接权重矩阵，与输入相乘生成一个隐层单元张量。若提供了’normalizer_fn’,则应用相关的正则化函数（如batch_norm）.另外，如果提供了‘biases_initializer’而没有提供‘normalizer_fn’,那么将生成‘biases’变量并加到隐层单元中去。最后，如果’activation_fn’不是‘None’，也会应用到隐层单元中去。

Args:

- inputs: A tensor of with at least rank 2 and value for the last dimension, i.e.

[batch_size, depth]

,

[None, None, None, channels]

.

- num_outputs: Integer or long, the number of output units in the layer.

- activation_fn: activation function, set to None to skip it and maintain a linear activation.

- normalizer_fn: normalization function to use instead of

biases

. If

normalizer_fn

is provided then

biases_initializer

and

biases_regularizer

are ignored and

biases

are not created nor added.default set to None for no normalizer function

- normalizer_params: normalization function parameters.

- weights_initializer: An initializer for the weights.

- weights_regularizer: Optional regularizer for the weights.

- biases_initializer: An initializer for the biases. If None skip biases.

- biases_regularizer: Optional regularizer for the biases

- reuse: whether or not the layer and its variables should be reused. To beable to reuse the layer scope must be given.

- variables_collections: Optional list of collections for all the variables or a dictionary containing a different list of collections per variable.

- outputs_collections: collection to add the outputs.

- t**rainable:** If

True

also add variables to the graph collection

GraphKeys.TRAINABLE_VARIABLES

(see tf.Variable).

- scope: Optional scope for variable_scope.

slim.conv2d_transpose()

convolution2d_transpose(

inputs,

num_outputs,

kernel_size,

stride=1,

padding=’SAME’,

data_format=DATA_FORMAT_NHWC,

activation_fn=nn.relu,

normalizer_fn=None,

normalizer_params=None,

weights_initializer=initializers.xavier_initializer(),

weights_regularizer=None,

biases_initializer=init_ops.zeros_initializer(),

biases_regularizer=None,

reuse=None,

variables_collections=None,

outputs_collections=None,

trainable=True,

scope=None)

The function creates a variable called

weights

, representing the kernel, that is convolved with the input.

Args:

inputs: A 4-D

Tensor

of type

float

and shape

[batch, height, width, in_channels]

for

NHWC

data format or

[batch, in_channels, height, width]

for

NCHW

data format.

num_outputs: integer, the number of output filters.

kernel_size: a list of length 2 holding the [kernel_height, kernel_width] of the filters. Can be an int if both values are the same.

stride: a list of length 2: [stride_height, stride_width].Can be an int if both strides are the same. Note that presently both strides must have the same value.

padding: one of ‘VALID’ or ‘SAME’.

data_format: A string.

NHWC

(default) and

NCHW

are supported.

activation_fn: activation function, set to None to skip it and maintain a linear activation.

normalizer_fn: normalization function to use instead of

biases

. If

normalizer_fn

is provided then

biases_initializer

and

biases_regularizer

are ignored and

biases

are not created nor added. default set to None for no normalizer function

normalizer_params: normalization function parameters.

weights_initializer: An initializer for the weights.

weights_regularizer: Optional regularizer for the weights.

biases_initializer: An initializer for the biases. If None skip biases.

biases_regularizer: Optional regularizer for the biases.

reuse: whether or not the layer and its variables should be reused. To be able to reuse the layer scope must be given.

variables_collections: optional list of collections for all the variables or a dictionary containing a different list of collection per variable.

outputs_collections: collection to add the outputs.

trainable: whether or not the variables should be trainable or not.

scope: Optional scope for variable_scope.

slim.flatten(inputs,outputs_collections=None,scope=None)

(注：import tensorflow.contrib.slim as slim)

将输入扁平化但保留batch_size,假设第一维是batch。

Args:

inputs: a tensor of size [batch_size, …].

outputs_collections: collection to add the outputs.

scope: Optional scope for name_scope.

如：

test=([[[1,2,3],[4,5,6],[7,8,9]],[[10,11,12],[13,14,15],[16,17,27]],[[18,19,20],[21,22,23],[24,25,26]]])    #shape is (3,3,3)
test=slim.fatten(test)
test.eval()
array([[ 1,  2,  3, ...,  7,  8,  9],
[10, 11, 12, ..., 16, 17, 27],
[18, 19, 20, ..., 24, 25, 26]], dtype=int32)
test.get_shape()
TensorShape([Dimension(3), Dimension(9)])  #(3,9)

tf.train.AdamOptimizer()

Adam，这是一种基于一阶梯度来优化随机目标函数的算法。随即目标函数的含义是，在训练过程的每一次迭代中，目标函数是不一样的。有时候因为内存不够大或者其他的原因，算法不会一下子读取全部记录来计算误差，而是选择选择对数据集进行分割，在每次迭代中只读取一部分记录进行训练，这一部分记录称为minibatch，这样每次迭代所使用的小批量数据集就是不同的，数据集不同，损失函数就不同，因此就有随机目标函数的说法。另外还有一个原因就是，采用小批量方式来进行训练，可以降低收敛到局部最优的风险.

Adam 这个名字来源于 adaptive moment estimation，自适应矩估计。概率论中矩的含义是：如果一个随机变量 X 服从某个分布，X 的一阶矩是 E(X)，也就是样本平均值，X 的二阶矩就是 E(X^2)，也就是样本平方的平均值。Adam 算法根据损失函数对每个参数的梯度的一阶矩估计和二阶矩估计动态调整针对于每个参数的学习速率。Adam 也是基于梯度下降的方法，但是每次迭代参数的学习步长都有一个确定的范围，不会因为很大的梯度导致很大的学习步长，参数的值比较稳定。

初始化Adam优化器：

tf.train.AdamOptimizer.init(learning_rate=0.001,beta1=0.9, beta2=0.999, epsilon=1e-08,use_locking=False, name=’Adam’)

待续