tensorflow variable_scope\name_scope
2017-11-14 16:11
423 查看
转载自http://blog.csdn.net/u011636567/article/details/71124211
转载自http://blog.csdn.net/qq_19918373/article/details/69499091
为了研究一下tensorflow的
1.对Variable先加
2
3
4
5
6
7
8
9
10
2.对Variable先加
2
3
4
5
6
7
8
3.对Summary先加
2
3
4
5
6
7
8
9
4.对Summary先加
还有需要注意一点的是
代码1
2
3
4
5
6
代码2
主要针对 tf.get_variable 来介绍共享变量的用法。
tf.get_variable 与 tf.variable 的用法不同。前者在创建变量时会查名字,如果给的名字在之前已经被别的变量占用,则会报错,不会创建相应变量。而后者并不进行检查,如果有重复,则自动的修改名字,加上数字来进行区别。所以从这来看要想共享变量并不能通过使用相同的名字来调用多次 tf.get_variable
和 tf.variable 做到。
比如下面这样的代码:
[python] view
plain copy
<span style="font-size:14px;">def my_image_filter(input_images):
conv1_weights = tf.Variable(tf.random_normal([5, 5, 32, 32]),
name="conv1_weights")
conv1_biases = tf.Variable(tf.zeros([32]), name="conv1_biases")
conv1 = tf.nn.conv2d(input_images, conv1_weights,
strides=[1, 1, 1, 1], padding='SAME')
relu1 = tf.nn.relu(conv1 + conv1_biases)
conv2_weights = tf.Variable(tf.random_normal([5, 5, 32, 32]),
name="conv2_weights")
conv2_biases = tf.Variable(tf.zeros([32]), name="conv2_biases")
conv2 = tf.nn.conv2d(relu1, conv2_weights,
strides=[1, 1, 1, 1], padding='SAME')
return tf.nn.relu(conv2 + conv2_biases)</span>
在这个函数中,我们有 'conv1_weights','conv1_biases','conv2_weights','conv2_biases'
4个变量。如果我们重用这个函数,则会产生多组变量,并不会使用相同的变量,如下面调用:
[python] view
plain copy
<span style="font-size:14px;"># First call creates one set of variables.
result1 = my_image_filter(image1)
# Another set is created in the second call.
result2 = my_image_filter(image2)</span>
上面实际上用两个不同的滤波器对 image1 和 image2 进行滤波,虽然用的是相同的函数。所以呢,这就产生了问题,下面介绍如何进行变量共享。
我们使用 with tf.variable_scope 来进行共享。比如有下面的代码:
[python] view
plain copy
<span style="font-size:14px;">def conv_relu(input, kernel_shape, bias_shape):
# Create variable named "weights".
weights = tf.get_variable("weights", kernel_shape,
initializer=tf.random_normal_initializer())
# Create variable named "biases".
biases = tf.get_variable("biases", bias_shape,
initializer=tf.constant_intializer(0.0))
conv = tf.nn.conv2d(input, weights,
strides=[1, 1, 1, 1], padding='SAME')
return tf.nn.relu(conv + biases)
def my_image_filter(input_images):
with tf.variable_scope("conv1"):
# Variables created here will be named "conv1/weights", "conv1/biases".
relu1 = conv_relu(input_images, [5, 5, 32, 32], [32])
with tf.variable_scope("conv2"):
# Variables created here will be named "conv2/weights", "conv2/biases".
return conv_relu(relu1, [5, 5, 32, 32], [32])</span>
若要调用两次 my_image_filter 并且使用相同的变量,则如下所示:
[python] view
plain copy
<span style="font-size:14px;">with tf.variable_scope("image_filters") as scope:
result1 = my_image_filter(image1)
scope.reuse_variables()
result2 = my_image_filter(image2)</span>
利用 reuse_variables() 来使变量重用。值得注意的是下面的代码解释了 tf.get_variable 工作原理:
[python] view
plain copy
<span style="font-size:14px;">with tf.variable_scope("foo"):
v = tf.get_variable("v", [1])
with tf.variable_scope("foo", reuse=True):
v1 = tf.get_variable("v", [1])
assert v1 == v</span>
如果 reuse 开启,当检查到有相同的名字时,直接返回那个有相同名字的变量而不是重新定义一个再复制值。
下面是使用时需要注意的地方
1. 在 variable_scope 里面的 variable_scope 会继承上面的 reuse 值,即上面一层开启了 reuse ,则下面的也跟着开启。但是不能人为的设置 reuse 为 false ,只有退出 variable_scope 才能让 reuse 变为 false:
[python] view
plain copy
<span style="font-size:14px;">with tf.variable_scope("root"):
# At start, the scope is not reusing.
assert tf.get_variable_scope().reuse == False
with tf.variable_scope("foo"):
# Opened a sub-scope, still not reusing.
assert tf.get_variable_scope().reuse == False
with tf.variable_scope("foo", reuse=True):
# Explicitly opened a reusing scope.
assert tf.get_variable_scope().reuse == True
with tf.variable_scope("bar"):
# Now sub-scope inherits the reuse flag.
assert tf.get_variable_scope().reuse == True
# Exited the reusing scope, back to a non-reusing one.
assert tf.get_variable_scope().reuse == False</span>
2. 当在某一 variable_scope 内使用别的 scope 的名字时,此时不再受这里的等级关系束缚,直接与使用的 scope 的名字一样:
[python] view
plain copy
<span style="font-size:14px;">with tf.variable_scope("foo") as foo_scope:
assert foo_scope.name == "foo"
with tf.variable_scope("bar")
with tf.variable_scope("baz") as other_scope:
assert other_scope.name == "bar/baz"
with tf.variable_scope(foo_scope) as foo_scope2:
assert foo_scope2.name == "foo" # Not changed.</span>
3. name_scope 与 variable_scope 稍有不同。name_scope 只会影响 ops 的名字,而并不会影响 variables 的名字。
[python] view
plain copy
<span style="font-size:14px;">with tf.variable_scope("foo"):
with tf.name_scope("bar"):
v = tf.get_variable("v", [1])
x = 1.0 + v
assert v.name == "foo/v:0"
assert x.op.name == "foo/bar/add"</span>
转载自http://blog.csdn.net/qq_19918373/article/details/69499091
为了研究一下tensorflow的
name_scope和
variable_scope到底有啥区别,我对Variable和Summary对象分别试验了这两种scope。直接上代码:
1.对Variable先加
name_scope,再加
variable_scope
import tensorflow as tf with tf.name_scope('ns1'): #name_scope v1 = tf.get_variable('v1', shape=(1,)) with tf.variable_scope('vs1'): #variable_scope v2 = tf.get_variable('v2', shape=(1,)) v1v2_coll = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES,scope='ns1') #[] v2_coll = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES,scope='vs1') #'vs1/v2:0' #name_scope对变量无效, 'vs1/v2:0'的prefix里没有'ns1/'1
2
3
4
5
6
7
8
9
10
2.对Variable先加
variable_scope,再加
name_scope
with tf.variable_scope('vs2'): #variable_scope v1 = tf.get_variable('v1', shape=(1,)) with tf.name_scope('ns2'): #name_scope v2 = tf.get_variable('v2', shape=(1,)) v1v2_coll = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES,scope='vs2') #'vs2/v1:0' v2_coll = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES,scope='ns2') #[] #name_scope对变量无效, 'vs2/v1:0'的prefix里没有'ns2/'1
2
3
4
5
6
7
8
3.对Summary先加
name_scope,再加
variable_scope
with tf.name_scope('ns3'): #name_scope tf.summary.histogram('sum_ns', tf.convert_to_tensor([1])) with tf.variable_scope('vs3'): #variable_scope tf.summary.histogram('sum_nsvs', tf.convert_to_tensor([1])) sum_ns_coll = tf.get_collection(tf.GraphKeys.SUMMARIES,scope='ns3') #'ns3/sum_ns:0', 'ns3/vs3/sum_vs:0' sum_nsvs_coll = tf.get_collection(tf.GraphKeys.SUMMARIES,scope='ns3/vs3') #'ns3/vs3/sum_nsvs:0'1
2
3
4
5
6
7
8
9
4.对Summary先加
variable_scope,再加
name_scope
with tf.variable_scope('vs4'): #variable_scope tf.summary.histogram('sum_vs', tf.convert_to_tensor([1])) with tf.name_scope('ns4'): #name_scope tf.summary.histogram('sum_ns', tf.convert_to_tensor([1])) sum_vs_coll = tf.get_collection(tf.GraphKeys.SUMMARIES,scope='vs4') #'vs4/sum_vs:0', 'vs4/ns4/sum_ns:0' sum_vsns_coll = tf.get_collection(tf.GraphKeys.SUMMARIES,scope='vs4/ns4') #'vs4/ns4/sum_ns:0'
还有需要注意一点的是
tf.variable_scope("name")与
tf.variable_scope(scope)的区别,看下面代码
代码1
import tensorflow as tf with tf.variable_scope("scope"): tf.get_variable("w",shape=[1])#这个变量的name是 scope/w with tf.variable_scope("scope"): tf.get_variable("w", shape=[1]) #这个变量的name是 scope/scope/w # 这两个变量的名字是不一样的,所以不会产生冲突1
2
3
4
5
6
代码2
import tensorflow as tf with tf.variable_scope("yin"): tf.get_variable("w",shape=[1]) scope = tf.get_variable_scope()#这个变量的name是 scope/w with tf.variable_scope(scope):#这种方式设置的scope,是用的外部的scope tf.get_variable("w", shape=[1])#这个变量的name也是 scope/w # 两个变量的名字一样,会报错
主要针对 tf.get_variable 来介绍共享变量的用法。
tf.get_variable 与 tf.variable 的用法不同。前者在创建变量时会查名字,如果给的名字在之前已经被别的变量占用,则会报错,不会创建相应变量。而后者并不进行检查,如果有重复,则自动的修改名字,加上数字来进行区别。所以从这来看要想共享变量并不能通过使用相同的名字来调用多次 tf.get_variable
和 tf.variable 做到。
比如下面这样的代码:
[python] view
plain copy
<span style="font-size:14px;">def my_image_filter(input_images):
conv1_weights = tf.Variable(tf.random_normal([5, 5, 32, 32]),
name="conv1_weights")
conv1_biases = tf.Variable(tf.zeros([32]), name="conv1_biases")
conv1 = tf.nn.conv2d(input_images, conv1_weights,
strides=[1, 1, 1, 1], padding='SAME')
relu1 = tf.nn.relu(conv1 + conv1_biases)
conv2_weights = tf.Variable(tf.random_normal([5, 5, 32, 32]),
name="conv2_weights")
conv2_biases = tf.Variable(tf.zeros([32]), name="conv2_biases")
conv2 = tf.nn.conv2d(relu1, conv2_weights,
strides=[1, 1, 1, 1], padding='SAME')
return tf.nn.relu(conv2 + conv2_biases)</span>
在这个函数中,我们有 'conv1_weights','conv1_biases','conv2_weights','conv2_biases'
4个变量。如果我们重用这个函数,则会产生多组变量,并不会使用相同的变量,如下面调用:
[python] view
plain copy
<span style="font-size:14px;"># First call creates one set of variables.
result1 = my_image_filter(image1)
# Another set is created in the second call.
result2 = my_image_filter(image2)</span>
上面实际上用两个不同的滤波器对 image1 和 image2 进行滤波,虽然用的是相同的函数。所以呢,这就产生了问题,下面介绍如何进行变量共享。
我们使用 with tf.variable_scope 来进行共享。比如有下面的代码:
[python] view
plain copy
<span style="font-size:14px;">def conv_relu(input, kernel_shape, bias_shape):
# Create variable named "weights".
weights = tf.get_variable("weights", kernel_shape,
initializer=tf.random_normal_initializer())
# Create variable named "biases".
biases = tf.get_variable("biases", bias_shape,
initializer=tf.constant_intializer(0.0))
conv = tf.nn.conv2d(input, weights,
strides=[1, 1, 1, 1], padding='SAME')
return tf.nn.relu(conv + biases)
def my_image_filter(input_images):
with tf.variable_scope("conv1"):
# Variables created here will be named "conv1/weights", "conv1/biases".
relu1 = conv_relu(input_images, [5, 5, 32, 32], [32])
with tf.variable_scope("conv2"):
# Variables created here will be named "conv2/weights", "conv2/biases".
return conv_relu(relu1, [5, 5, 32, 32], [32])</span>
若要调用两次 my_image_filter 并且使用相同的变量,则如下所示:
[python] view
plain copy
<span style="font-size:14px;">with tf.variable_scope("image_filters") as scope:
result1 = my_image_filter(image1)
scope.reuse_variables()
result2 = my_image_filter(image2)</span>
利用 reuse_variables() 来使变量重用。值得注意的是下面的代码解释了 tf.get_variable 工作原理:
[python] view
plain copy
<span style="font-size:14px;">with tf.variable_scope("foo"):
v = tf.get_variable("v", [1])
with tf.variable_scope("foo", reuse=True):
v1 = tf.get_variable("v", [1])
assert v1 == v</span>
如果 reuse 开启,当检查到有相同的名字时,直接返回那个有相同名字的变量而不是重新定义一个再复制值。
下面是使用时需要注意的地方
1. 在 variable_scope 里面的 variable_scope 会继承上面的 reuse 值,即上面一层开启了 reuse ,则下面的也跟着开启。但是不能人为的设置 reuse 为 false ,只有退出 variable_scope 才能让 reuse 变为 false:
[python] view
plain copy
<span style="font-size:14px;">with tf.variable_scope("root"):
# At start, the scope is not reusing.
assert tf.get_variable_scope().reuse == False
with tf.variable_scope("foo"):
# Opened a sub-scope, still not reusing.
assert tf.get_variable_scope().reuse == False
with tf.variable_scope("foo", reuse=True):
# Explicitly opened a reusing scope.
assert tf.get_variable_scope().reuse == True
with tf.variable_scope("bar"):
# Now sub-scope inherits the reuse flag.
assert tf.get_variable_scope().reuse == True
# Exited the reusing scope, back to a non-reusing one.
assert tf.get_variable_scope().reuse == False</span>
2. 当在某一 variable_scope 内使用别的 scope 的名字时,此时不再受这里的等级关系束缚,直接与使用的 scope 的名字一样:
[python] view
plain copy
<span style="font-size:14px;">with tf.variable_scope("foo") as foo_scope:
assert foo_scope.name == "foo"
with tf.variable_scope("bar")
with tf.variable_scope("baz") as other_scope:
assert other_scope.name == "bar/baz"
with tf.variable_scope(foo_scope) as foo_scope2:
assert foo_scope2.name == "foo" # Not changed.</span>
3. name_scope 与 variable_scope 稍有不同。name_scope 只会影响 ops 的名字,而并不会影响 variables 的名字。
[python] view
plain copy
<span style="font-size:14px;">with tf.variable_scope("foo"):
with tf.name_scope("bar"):
v = tf.get_variable("v", [1])
x = 1.0 + v
assert v.name == "foo/v:0"
assert x.op.name == "foo/bar/add"</span>
相关文章推荐
- tensorflow scope命名方法(variable_scope()与name_scope()解析)
- What's the difference of name scope and a variable scope in tensorflow?
- tensorflow variable_scope,tf.name_scope, tf.variable, tf.get_varible
- 转载!tensorflow name scope和variable scope
- tensorflow name_scope与variable_scope
- What's the difference of name scope and a variable scope in tensorflow?
- tensorflow variable scope 变量命名空间和变量共享
- Notes on tensorflow(六)variable_scope
- ValueError: Attempt to reuse RNNCell <tensorflow.contrib.rnn.python.ops.core_rnn_cell_impl.BasicLSTMCell object at 0x7f1a3c448390> with a different variable scope than its first use.解决方法
- tensorflow name scope怎么让tensorboard显示更加clean
- tensorflow variable_scope共享变量
- TensorFlow 学习(一)—— tf.get_variable() vs tf.Variable(),tf.name_scope() vs tf.variable_scope()
- 【Tensorflow】name_scope() 和 variable_scope() 区别
- Tensorflow name_scope和variable_scope的区别
- 彻底弄懂tf.Variable、tf.get_variable、tf.variable_scope以及tf.name_scope异同
- tensorflow学习笔记(十七):name&variable scope
- tf.name_scope and tf.variable_scope
- TensorFlow Session&&Variable&&PlaceHolder
- tf.name_scope() 与 tf.variable_scope() 的区别
- TensorFlow入门(七) 充分理解 name / variable_scope