openstack快照分析

1.openstack快照

1.1.OpenStack快照类别

注意：不要混淆为卷创建快照和为以卷作为后端存储的虚拟机创建快照

虚拟机快照（快照存放于glance）：nova
image-create <server> <name>

cold snapshot:创建snapshot时，需暂停虚拟机。
live snapshot with no downtime:创建snapshot时，无需暂停虚拟机。（目前未实现，存在的问题见此链接：http://blog.chinaunix.net/uid-29718549-id-4346700.html）

volume backed以外类型的虚拟机，在nova-api中调用API.snapshot接口创建快照（最终实现在createImage接口）
volume backed的虚拟机，在nova-api中调用API.snapshot_volume_backed接口创建虚拟机快照

卷快照（LVM快照存放于本地）:
nova volume-snapshot-create <volume-id>
在nova-api中调用API.volume_snapshot_create接口创建虚拟机快照

1.2.openstack创建快照命令

nova image-create

usage: nova image-create [--show] [--poll] <server> <name>

Create a new image by taking a snapshot of a running server.

Positional arguments
<server>
Name or ID of server.
<name>
Name of snapshot.

Optional arguments

--show
Print image info.

--poll
Report the snapshot progress and poll until image creation is complete.

nova volume-snapshot-create

usage: nova volume-snapshot-create [--force <True|False>]
[--display-name <display-name>]
[--display-description <display-description>]
<volume-id>

Add a new snapshot.

Positional arguments
<volume-id>
ID of the volume to snapshot

Optional arguments
--force <True|False>
Optional flag to indicate whether to snapshot a volume even if its attached to a server. (Default=False)
--display-name <display-name>
Optional snapshot name. (Default=None)
--display-description <display-description>
Optional snapshot description. (Default=None)

1.3.流程

cold snapshot 流程:

# Save the state and stop a running guest, then detach pci devices
$ virsh managedsave vm (managedsave流程见此链接：http://blog.chinaunix.net/uid-29718549-id-4772180.html)
# Create a qemu internal snapshot
$ qemu-img snapshot -c snap1 vm_snapshot
# Extract the internal snapshot, convert it to qcow2 and export it a file, then upload to glance
$ qemu-img convert -f qcow2 vm -O qcow2 vm_snapshot
# Start the guest again
$ virsh start vm

live snapshot 流程:

# Abort any failed/finished block operations:

$ virsh blockjob vm vda --abort

# Undefine a running domain. (Note: Undefining a running domain does not _kill_ the domain, it just converts it from persistent to transient.)

$ virsh undefine vm

# create a destination image with the original backing file and matching size of the instance root disk.

$ qemu-img create -f qcow2 vm_copy --backing_file=backing_file --size=root_disk_size

#Invoke 'virsh blockcopy' (This will take time, depending on the size of disk image vm1):

$ virsh blockcopy --domain vm vda vm_copy --wait --verbose

#Abort any failed/finished block operations:

$ virsh blockjob vm vda --abort

#Define the guest again (to make it persistent):

$ virsh define vm

#From the obtained new copy, convert the QCOW2 with a backing file to a qcow2 image with no backing file, then upload to glance:

$ qemu-img convert -f qcow2 -O raw vm_copy vm_convert

2.与传统快照的区别

对openstack而言，虚拟机的快照其实就是镜像。这是因为Openstack是采用创建镜像的方式创建快照（qemu-img
convert -f qcow2 -O qcow2 <disk_path> <out_path>），而不是通过正统的virsh或者qemu的live snapshot创建快照函数( virsh
snapshot-create-as --live ....)来实现的。虚拟机快照做完后以镜像形式存于glance。openstack的快照与传统快照有以下区别：

libvirt 主流快照实现： 采用virDomainSnapshotCreateXML()函数(CLI为virsh snapshot-create)。新建的快照与虚拟机有关联：若为内置快照，快照信息和虚拟机存在同一个qcow2镜像中；若为外置快照，新建一个qcow2文件，原虚拟机的disk将变为一个read
only的模板镜像，新qcow2镜像仅记录与2.模板镜像的差异数据。这种快照包含快照链信息，可保留disk和ram信息，可回滚至快照点。
openstack快照实现：openstack并未采用virDomainSnapshotCreateXML()来实现快照，而是单纯的对虚拟机镜像做转换和拷贝，生成一个与虚拟机无关联的镜像，最后上传至glance中。这种快照不包含快照链信息，只保留disk信息，无法回滚至快照点，只能采用该快照镜像创建一个新的虚拟机。

限制与缺点：

没有快照链信息，不支持revert恢复虚拟机到某一个快照点
只对系统盘进行快照，
不支持内存快照，不支持同时对虚拟机和磁盘做快照
需要用户进行一致性操作

不支持含元数据导出，

不支持含元数据导入

只支持虚拟机全量数据快照（与快照的实现方式有关，因为是通过image进行保存的）
过程较长（需要先通过存储快照，然后抽取并上传至glance)
快照以Image方式保存，而非以cinder卷方式保存，无法充分利用存储本身能力加快快照的创建和使用
当前限制openstack的虚拟机快照只快照root盘，不快照内存/CPU状态以及挂载磁盘。挂载磁盘需要事先卸载磁盘(数据盘），然后进行快照，然后再挂载磁盘
没有快照链信息

3.OpenStack快照源代码流程分析

3.1 nova/compute/api.py

# NOTE(melwitt): We
don't check instance lock for snapshot because lock is

# intended to prevent accidental change/delete of instances

@wrap_check_policy

@check_instance_cell

@check_instance_state(vm_state=[vm_states.ACTIVE, vm_states.STOPPED,

vm_states.PAUSED, vm_states.SUSPENDED])

def snapshot(self, context, instance, name, extra_properties=None):

"""Snapshot the given instance.

:param instance: nova.db.sqlalchemy.models.Instance

:param name: name of the snapshot

:param extra_properties: dict of extra image properties to include

when creating the image.

:returns: A dict containing image metadata

"""

#_create_image函数中调用glance
api创建image entry，为后将snapshot上传为镜像做准备，虽然镜像和snapshot在可以上传到glance作为镜像启动虚拟机，

#但是为了区分二者的不同，glance将镜像和snapshot标记为不同的类型：type=image 和 type=snapshot

image_meta = self._create_image(context, instance, name,

'snapshot',

extra_properties=extra_properties)

# NOTE(comstud): Any changes to this method should also be made

# to the snapshot_instance() method in nova/cells/messaging.py

# 将任务状态(task state) 设置为:image_snapshot_pending

instance.task_state = task_states.IMAGE_SNAPSHOT_PENDING

instance.save(expected_task_state=[None])

#通过rpc调用nova/compute/rpcapi.py的snapshot_instance函数

self.compute_rpcapi.snapshot_instance(context, instance,

image_meta['id

3.2 nova/compute/rpcapi.py

#梳理下流程：

（1）用户发起create snapshot的请求;

（2）nova-api服务接收到这个请求并进行前期处理，即3.1中代码的处理流程;

（3）真正的snapshot操作是需要在nova-compute节点上执行的，所以nova-api需要向nova-compute发送message

#由于OpenStack环境中会有很多个nova-compute,所以需要通过server=_compute_host(None, instance)来获取虚拟机所在的host，并向其发送message。

def snapshot_instance(self, ctxt, instance, image_id):

version = '3.0'

cctxt = self.client.prepare(server=_compute_host(None, instance),

version=version)

cctxt.cast(ctxt, 'snapshot_instance',

instance=instance,

image_id=image_id)

3.3 nova/virt/libvirt/driver.py

def snapshot(self, context, instance, image_id, update_task_state):

"""Create snapshot from a running VM instance.

This command only works with qemu 0.14+

"""

try:

virt_dom = self._get_domain(instance)

except exception.InstanceNotFound:

raise exception.InstanceNotRunning(instance_id=instance['uuid'])

base_image_ref = instance['image_ref']

base = compute_utils.get_image_metadata(

context, self._image_api, base_image_ref, instance)

snapshot = self._image_api.get(context, image_id)

disk_path = libvirt_utils.find_disk(virt_dom)

source_format = libvirt_utils.get_disk_type(disk_path)

image_format = CONF.libvirt.snapshot_image_format or source_format

# NOTE(bfilippov): save lvm and rbd
as raw

if image_format == 'lvm' or image_format == 'rbd':

image_format = 'raw'

metadata = self._create_snapshot_metadata(base,

instance,

image_format,

snapshot['name'])

snapshot_name = uuid.uuid4().hex

state = LIBVIRT_POWER_STATE[virt_dom.info()[0]]

# NOTE(rmk): Live
snapshots require QEMU 1.3 and Libvirt 1.0.0.

# These restrictions can be relaxed as other configurations

# can be validated.

# NOTE(dgenin): Instances with LVM
encrypted ephemeral storage require

# cold snapshots. Currently, checking for encryption is

# redundant because LVM supports only cold snapshots.

# It is necessary in case this situation changes in the

# future.

#这里需要注意，解释了为啥现在是cold snapshot而不是live snapshot:

# 有人提过live snapshot的bug，社区认为live snapshot目前不稳定，所以默认条件下采用cold snapshot，并且是通过硬编码来实现的

#
看下面这个判断条件，成立的时候将live_snapshot = true,其中MIN_LIBVIRT_LIVESNAPSHOT_VERSION=1.3.0, 其实现在libvirt的最新版本

# 才到1.2.11， 所以这个live_snapshot的条件不满足，就变成了cold_snapshot

if (self._host.has_min_version(MIN_LIBVIRT_LIVESNAPSHOT_VERSION,

MIN_QEMU_LIVESNAPSHOT_VERSION,

REQ_HYPERVISOR_LIVESNAPSHOT)

and source_format not in ('lvm', 'rbd')

and not CONF.ephemeral_storage_encryption.enabled):

live_snapshot = True

# Abort is an idempotent operation, so make sure any block

# jobs which may have failed are ended. This operation also

# confirms the running instance, as opposed to the system as a

# whole, has a new enough version of the hypervisor (bug 1193146).

try:

virt_dom.blockJobAbort(disk_path, 0)#live第一步

except libvirt.libvirtError as ex:

error_code = ex.get_error_code()

if error_code == libvirt.VIR_ERR_CONFIG_UNSUPPORTED:

live_snapshot = False

else:

pass

else:

live_snapshot = False

# NOTE(rmk): We cannot perform live
snapshots when a managedSave

# file is present, so we will use the cold/legacy
method

# for instances which are shutdown.

if state == power_state.SHUTDOWN:

live_snapshot = False

# NOTE(dkang): managedSave does not work for LXC

#注意这里，如果live_snashot目前是false,在做snapshot之前先要执行：

#（1）_detach_pci_devices，
卸载虚拟机挂载的pci设备，比如数据盘

#（2）self._detach_sriov_ports，
卸载虚拟机挂载的SRIOV设备，比如支持SRIOV的网卡设备

if CONF.libvirt.virt_type != 'lxc' and not live_snapshot:

if state == power_state.RUNNING or state == power_state.PAUSED:

self._detach_pci_devices(virt_dom,

pci_manager.get_instance_pci_devs(instance))

self._detach_sriov_ports(instance, virt_dom)

virt_dom.managedSave(0)#cold第一步

#判断虚拟机的后端存储是什么，不同的后端存储做snapshot是不同的，本地文件系统的话，默认qcow2

snapshot_backend = self.image_backend.snapshot(instance,

disk_path,

image_type=source_format)

if live_snapshot:

LOG.info(_LI("Beginning
live snapshot process"),

instance=instance)

else:

LOG.info(_LI("Beginning
cold snapshot process"),

instance=instance)

#更新任务的状态为：image_pending_upload, 大家都知道做完snapshot要上传

update_task_state(task_state=task_states.IMAGE_PENDING_UPLOAD)

#目前做快照的过程是：

#（1）先在../data/nova/instance/snapshots目录下生成临时目录，比如nova/instances/snapshots/tmptHr585

#（2）然后将快照生成到这个目录，具体参见snapshot_backend.snapshot_extract(out_path, image_format)这个函数

#（3）生成完成后，通过glance api上传，具体参见 self._image_api.update

snapshot_directory = CONF.libvirt.snapshots_directory

fileutils.ensure_tree(snapshot_directory)

with utils.tempdir(dir=snapshot_directory) as
tmpdir:

try:

out_path = os.path.join(tmpdir, snapshot_name)

if live_snapshot:

# NOTE(xqueralt): libvirt
needs o+x in the temp directory

os.chmod(tmpdir, 0o701)

self._live_snapshot(context, instance, virt_dom, disk_path,

out_path, image_format, base)#热迁移

else:

#这个函数实际执行了一条命令：
qemu-img convert -f qcow2 -O qcow2 disk_path out_path,算是生成了快照

snapshot_backend.snapshot_extract(out_path, image_format)

finally:

new_dom = None

# NOTE(dkang): because previous
managedSave is not called

# for LXC, _create_domain must not be
called.

if CONF.libvirt.virt_type != 'lxc' and not live_snapshot:

if state == power_state.RUNNING:

new_dom = self._create_domain(domain=virt_dom) ##恢复做快照之前虚拟机的状态

elif state == power_state.PAUSED:

new_dom = self._create_domain(domain=virt_dom,

launch_flags=libvirt.VIR_DOMAIN_START_PAUSED)

if new_dom is not None:

self._attach_pci_devices(new_dom,

pci_manager.get_instance_pci_devs(instance))

self._attach_sriov_ports(context, instance, new_dom)

LOG.info(_LI("Snapshot
extracted, beginning image upload"),

instance=instance)

# Upload that image to the image service

update_task_state(task_state=task_states.IMAGE_UPLOADING,

expected_state=task_states.IMAGE_PENDING_UPLOAD)

with libvirt_utils.file_open(out_path) as
image_file: ###将生成的快照上传到glance

self._image_api.update(context,

image_id,

metadata,

image_file)

LOG.info(_LI("Snapshot
image upload complete"),

instance=instance)

参考资料：
1. https://www.redhat.com/archives/libvirt-users/2012-April/msg00142.html
2. /article/1667694.html
2. http://itxx.sinaapp.com/blog/content/130
3. http://kashyapc.com/2012/09/14/externaland-live-snapshots-with-libvirt/
4.http://blog.csdn.net/wsfdl/article/details/38522099
5./article/1667694.html
6.http://blog.chinaunix.net/uid-20940095-id-4752643.html