TensorFlow学习日记28

1. TensorFlow和文本，语音，图像和视频

解析：

（1）序列模型（seq2seq）

（2）语音（Pydub、Scipy、SpeechPy）

（3）视频（MoviePy）

（4）图像（TF-Slim，图像分割，目标检测，目标跟踪）

（5）性能分析（tfprof）

（6）代码调试（tfdbg）

（7）可视化工具（TensorBoard）

（8）Keras接口（tf.keras）

（9）Spark（TensorFlow On Spark）

（10）OCR（tf.nn.ctc_loss，warp-ctc）

2. TensorFlow和动态图机制Eager Execution

解析：当启动Eager Execution时，运算会即刻执行，无需Session.run()就可以把它们的值返回到Python。两个矩阵相乘的代码，如下所示：

import tensorflow as tf
import tensorflow.contrib.eager as tfe

tfe.enable_eager_execution()

x = [[2.]]
m = tf.matmul(x, x)
print(m)

3. FLAGS = tf.flags.FLAGS

解析： FLAGS是一个对象，保存了解析后的命令行参数。

4. FLAGS._parse_flags()

解析：将FLAGS.__flags转换为字典类型。

5. allow_soft_placement，log_device_placement

解析：sess = tf.Session(config=tf.ConfigProto(allow_soft_placement=True, log_device_placement=True))

6. RNNCell，BasicRNNCell和BasicLSTMCell

解析：

（1）RNNCell是一个抽象类。

（2）BasicRNNCell是RNN的基础类。

（3）BasicLSTMCell是LSTM的基础类。

7. tf.app.run()

解析：处理flag解析，然后执行main函数。

8. tf.nn.embedding_lookup

解析：tf.nn.embedding_lookup(params,ids,partition_strategy='mod',name=None,validate_indices=True,max_norm=None)，其中embedding_lookup(params, ids)按照ids顺序返回params中的第ids行。比如，ids=[1,3,2]就是返回params中第1，3，2行。返回结果为由params的1，3，2行组成的tensor。

9. 数据格式(batch_size, time_steps, input_size)

解析：其中time_steps表示序列长度，input_size表示输入数据单个序列单个时间维度上的长度。

10. MultiRNNCell

解析：将x输入第一层RNN后得到隐层状态h，这个隐层状态就相当于第二层RNN的输入，第二层RNN的隐层状态又相当于第三层RNN的输入，以此类推。如下所示：

import tensorflow as tf
import numpy as np
def get_a_cell():
return tf.nn.rnn_cell.BasicRNNCell(num_units=128)
cell = tf.nn.rnn_cell.MultiRNNCell([get_a_cell() for _ in range(3)])
print(cell.state_size)
inputs = tf.placeholder(np.float32, shape=(32, 100))
h0 = cell.zero_state(32, np.float32)
output, h1 = cell.call(inputs, h0)
print(h1)

说明：cell的state_size是(128, 128, 128)，表示共有3个隐层状态，每个隐层状态的大小为128；h1的数据类型为tuple，它表示含有3个32x128的向量。

11. tf.nn.dynamic_rnn

解析：调用call只是在时间序列上前进一步，使用tf.nn.dynamic_rnn就相当于调用n次call函数。即通过{h0,x1,x2, …,xn}直接得{h1,h2,…,hn}。假设已经定义好一个RNNCell，调用该RNNCell的call函数time_steps次，如下所示：

# inputs: shape = (batch_size, time_steps, input_size)
# cell: RNNCell
# initial_state: shape = (batch_size, cell.state_size)
outputs, state = tf.nn.dynamic_rnn(cell, inputs, initial_state=initial_state)

说明：outputs是time_steps步中所有的输出，它的形状为(batch_size, time_steps, cell.output_size)。state是最后一步的隐状态，它的形状为(batch_size, cell.state_size)。

12. state_size和output_size

解析：前者表示隐层的大小，后者表示输出的大小。比如将一个batch输入模型计算，假如输入数据的形状为(batch_size, input_size)，那么计算时得到的隐层状态就是(batch_size, state_size)，输出就是(batch_size, output_size)。对于BasicLSTMCell，因为LSTM有两个隐状态h和c，对应的隐层就是一个Tuple，每个都是(batch_size, state_size)的形状。

（1）RNN情况，如下所示：

import tensorflow as tf
import numpy as np
cell = tf.nn.rnn_cell.BasicRNNCell(num_units=128)
print(cell.state_size) # 128
inputs = tf.placeholder(np.float32, shape=(32, 100))
h0 = cell.zero_state(32, np.float32)
output, h1 = cell.call(inputs, h0)
print(h1.shape)

（2）LSTM情况，如下所示：

import tensorflow as tf
import numpy as np
lstm_cell = tf.nn.rnn_cell.BasicLSTMCell(num_units=128)
inputs = tf.placeholder(np.float32, shape=(32, 100))
h0 = lstm_cell.zero_state(32, np.float32)
output, h1 = lstm_cell.call(inputs, h0)
print(h1.h)
print(h1.c)

13. tf.Variable

解析：trainable If True, the default, also adds the variable to the graph collection GraphKeys.TRAINABLE_VARIABLES. This collection is used as the default list of variables to use by the Optimizer classes.

14. tf.get_default_graph().as_graph_def()

解析：

（1）get_default_graph：Returns the default graph for the current thread.

（2）as_graph_def：Returns a serialized GraphDef representation of this graph.

15. random()

解析：random()方法返回随机生成的一个实数，它在[0,1)范围内。

16. tf.nn.embedding_lookup

解析：

import tensorflow as tf
import numpy as np
a = [[0.1, 0.2, 0.3], [1.1, 1.2, 1.3], [2.1, 2.2, 2.3], [3.1, 3.2, 3.3], [4.1, 4.2, 4.3]]
a = np.asarray(a)
idx1 = tf.Variable([0, 2, 3, 1], tf.int32)
idx2 = tf.Variable([[0, 2, 3, 1], [4, 0, 2, 2]], tf.int32)
out1 = tf.nn.embedding_lookup(a, idx1)
out2 = tf.nn.embedding_lookup(a, idx2)
init = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init)
print(sess.run(out1))
print(out1)
print("************")
print(sess.run(out2))
print(out2)

17. tf.expand_dims
解析：

# 't' is a tensor of shape [2]
shape(expand_dims(t, 0)) ==> [1, 2]
shape(expand_dims(t, 1)) ==> [2, 1]
shape(expand_dims(t, -1)) ==> [2, 1]
# 't2' is a tensor of shape [2, 3, 5]
shape(expand_dims(t2, 0)) ==> [1, 2, 3, 5]
shape(expand_dims(t2, 2)) ==> [2, 3, 1, 5]
shape(expand_dims(t2, 3)) ==> [2, 3, 5, 1]

18. tf.equal
解析：对应元素相同为True，不同为False。

19. tf.reduce_mean

解析：求平均值。

20. tf.reduce_max()

解析：求最大值。

21. DropoutWrapper

解析：

lstm = tf.contrib.rnn.BasicLSTMCell(lstm_hidden_size)
dropout_lstm = tf.contrib.rnn.DropoutWrapper(lstm, keep_prob=0.5)
stacked_lstm = tf.contrib.rnn.MultiRNNCell([dropout_lstm] * number_of_layers)

22. tf.contrib.rnn.static_rnn和tf.nn.static_rnn

解析： static_rnn(cell,inputs,initial_state=None,dtype=None,sequence_length=None,scope=None)，即通过cell创建一个RNN。如下所示：

（1）cell: An instance of RNNCell.

（2）inputs: A length T list of inputs, each a Tensor of shape [batch_size, input_size], or a nested tuple of such elements.

（3）initial_state: (optional) An initial state for the RNN. If cell.state_size is an integer, this must be a Tensor of appropriate type and shape [batch_size, cell.state_size]. If cell.state_size is a tuple, this should be a tuple
of tensors having shapes [batch_size, s] for s in cell.state_size.

（4）dtype: (optional) The data type for the initial state and expected output. Required if initial_state is not provided or RNN state has a heterogeneous dtype.

（5）sequence_length: Specifies the length of each sequence in inputs. An int32 or int64 vector (tensor) size [batch_size], values in [0, T).

（6）scope: VariableScope for the created subgraph; defaults to "rnn".

说明：outputs is a length T list of outputs (one for each input), or a nested tuple of such elements; state is the final state。

23. tf.nn.dynamic_rnn

解析：dynamic_rnn(cell, inputs, sequence_length=None, initial_state=None, dtype=None, parallel_iterations=None, swap_memory=False, time_major=False, scope=None)。如下所示： 

（1）创建1个BasicRNNCell

rnn_cell = tf.nn.rnn_cell.BasicRNNCell(hidden_size)
# 'outputs' is a tensor of shape [batch_size, max_time, cell_state_size]
# defining initial state
initial_state = rnn_cell.zero_state(batch_size, dtype=tf.float32)
# 'state' is a tensor of shape [batch_size, cell_state_size]
outputs, state = tf.nn.dynamic_rnn(rnn_cell, input_data,
initial_state=initial_state,
dtype=tf.float32)

（2）创建2个LSTMCells

rnn_layers = [tf.nn.rnn_cell.LSTMCell(size) for size in [128, 256]]
# create a RNN cell composed sequentially of a number of RNNCells
multi_rnn_cell = tf.nn.rnn_cell.MultiRNNCell(rnn_layers)
# 'outputs' is a tensor of shape [batch_size, max_time, 256]
# 'state' is a N-tuple where N is the number of LSTMCells containing a
# tf.contrib.rnn.LSTMStateTuple for each cell
outputs, state = tf.nn.dynamic_rnn(cell=multi_rnn_cell,
inputs=data,
dtype=tf.float32)

24. tf.contrib.rnn.static_bidirectional_rnn和tf.nn.static_bidirectional_rnn
解析：static_bidirectional_rnn(cell_fw, cell_bw,inputs, initial_state_fw=None, initial_state_bw=None, dtype=None, sequence_length=None, scope=None)。如下所示： 

（1）cell_fw: An instance of RNNCell, to be used for forward direction.

（2）cell_bw: An instance of RNNCell, to be used for backward direction.

（3）inputs: A length T list of inputs, each a tensor of shape [batch_size, input_size], or a nested tuple of such elements.

（4）initial_state_fw: (optional) An initial state for the forward RNN. This must be a tensor of appropriate type and shape [batch_size, （5）cell_fw.state_size]. If cell_fw.state_size is a tuple, this should be a tuple of tensors having shapes [batch_size, s]
for s in （6）cell_fw.state_size.

（5）initial_state_bw: (optional) Same as for initial_state_fw, but using the corresponding properties of cell_bw.

（6）dtype: (optional) The data type for the initial state. Required if either of the initial states are not provided.

（7）sequence_length: (optional) An int32/int64 vector, size [batch_size], containing the actual lengths for each of the sequences.

（8）scope: VariableScope for the created subgraph; defaults to "bidirectional_rnn".

25. tf.nn.bidirectional_dynamic_rnn

解析：bidirectional_dynamic_rnn(cell_fw, cell_bw, inputs, sequence_length=None, initial_state_fw=None, initial_state_bw=None, dtype=None, parallel_iterations=None, swap_memory=False, time_major=False, scope=None)。如下所示： 

（1）cell_fw: An instance of RNNCell, to be used for forward direction.

（2）cell_bw: An instance of RNNCell, to be used for backward direction.

（3）inputs: The RNN inputs. If time_major == False (default), this must be a tensor of shape: [batch_size, max_time, ...], or a nested tuple of such elements. If time_major == True, this must be a tensor of shape: [max_time, batch_size, ...], or a nested tuple
of such elements.

（4）sequence_length: (optional) An int32/int64 vector, size [batch_size], containing the actual lengths for each of the sequences in the batch. If not provided, all batch entries are assumed to be full sequences; and time reversal is applied from time 0 to max_time
for each sequence.

（5）initial_state_fw: (optional) An initial state for the forward RNN. This must be a tensor of appropriate type and shape [batch_size, cell_fw.state_size]. If cell_fw.state_size is a tuple, this should be a tuple of tensors having shapes [batch_size, s] for
s in cell_fw.state_size.

（6）initial_state_bw: (optional) Same as for initial_state_fw, but using the corresponding properties of cell_bw.

（7）dtype: (optional) The data type for the initial states and expected output. Required if initial_states are not provided or RNN states have a heterogeneous dtype.

（8）parallel_iterations: (Default: 32). The number of iterations to run in parallel. Those operations which do not have any temporal dependency and can be run in parallel, will be. This parameter trades off time for space. Values >> 1 use more memory but take
less time, while smaller values use less memory but computations take longer.

（9）swap_memory: Transparently swap the tensors produced in forward inference but needed for back prop from GPU to CPU. This allows training RNNs which would typically not fit on a single GPU, with very minimal (or no) performance penalty.

（10）time_major: The shape format of the inputs and outputs Tensors. If true, these Tensors must be shaped [max_time, batch_size, depth]. If false, these Tensors must be shaped [batch_size, max_time, depth]. Using time_major = True is a bit more efficient because
it avoids transposes at the beginning and end of the RNN calculation. However, most TensorFlow data is batch-major, so by default this function accepts input and emits output in batch-major form.

（11）scope: VariableScope for the created subgraph; defaults to "bidirectional_rnn".

参考文献：

[1] seq2seq：https://google.github.io/seq2seq/ 

[2] tf.nn.embedding_lookup：https://www.jianshu.com/p/ad88a0afa98f