块存储 Cinder磁盘加密方法 研究

在openstack havana的release note中有如下介绍“Attached Cinder volumes can now be encrypted. Data is decrypted as needed at read and write time while presenting instances with a normal block storage device”。

众所周知，在以前版本的openstack中，块存储创建出volume后，将其挂载给虚拟机，就可以启动虚拟机。但这种实现方式也存在一些安全隐患：若存在不怀好意的系统管理员或者黑客在攻陷nova节点时，就可以从宿主系统中，查看guest os中用户存放的数据。如果用户在其磁盘中存在敏感私人数据时，则存在数据泄密的风险。

关于磁盘加密的BluePrint链接：https://wiki.openstack.org/wiki/VolumeEncryption ，通过分析其实现方式可以发现，cinder中的卷在挂载到HyperVisor主机上时，对其进行加密，再讲加密后的块设备提供给虚拟机使用，最终虚拟机使用的加密后的磁盘。如果此时黑客攻陷主机的话，他看到的将是加密后的volume，如果不知道磁盘的加密密码，他看到将是没有任何意义的一堆数据。



通过分析nova中代码实现，可以发现磁盘加密的实现细节，openstack中的磁盘加密使用了Linux中的一个加密组件：cryptsetup！

[python] view
plain copy







def _open_volume(self, passphrase, **kwargs):

"""Opens the LUKS partition on the volume using the specified

passphrase.

:param passphrase: the passphrase used to access the volume

"""

LOG.debug(_("opening encrypted volume %s"), self.dev_path)

# NOTE(joel-coffman): cryptsetup will strip trailing newlines from

# input specified on stdin unless --key-file=- is specified.

cmd = ["cryptsetup", "create", "--key-file=-"]

cipher = kwargs.get("cipher", None)

if cipher is not None:

cmd.extend(["--cipher", cipher])

key_size = kwargs.get("key_size", None)

if key_size is not None:

cmd.extend(["--key-size", key_size])

cmd.extend([self.dev_name, self.dev_path])

utils.execute(*cmd, process_input=passphrase,

check_exit_code=True, run_as_root=True)

def attach_volume(self, context, **kwargs):

"""Shadows the device and passes an unencrypted version to the

instance.

Transparent disk encryption is achieved by mounting the volume via

dm-crypt and passing the resulting device to the instance. The

instance is unaware of the underlying encryption due to modifying the

original symbolic link to refer to the device mounted by dm-crypt.

"""

key = self._get_key(context).get_encoded()

passphrase = self._get_passphrase(key)

self._open_volume(passphrase, **kwargs)

# modify the original symbolic link to refer to the decrypted device

utils.execute('ln', '--symbolic', '--force',

'/dev/mapper/%s' % self.dev_name, self.symlink_path,

run_as_root=True, check_exit_code=True)

可见，在attach_volume方法的_open_volume中，会调用cryptsetup系统命令对输入的待挂载设备进行加密，再将加密的设备生成挂载给虚拟机的软链接，这样虚拟机在启动时感知不到加密方法的存在，即磁盘加密对虚拟机是透明的。cryptsetup

由于手头没有havana的环境，下面打算根据cinder的实现原理对磁盘加密进行模拟，同时研究其实现：

1、首先使用truncate命令生成一个空洞文件

[plain] view
plain copy







[root@armstrong tmp]# truncate --size=10G 10G

[root@armstrong tmp]# ll /tmp/10G

-rw-r--r--. 1 root root 10737418240 12月 3 23:37 /tmp/10G

[root@armstrong tmp]#

2、将上述空洞文件转化为loop设备

[plain] view
plain copy







[root@armstrong tmp]#

[root@armstrong tmp]# losetup -f /tmp/10G

[root@armstrong tmp]#

[root@armstrong tmp]# losetup -a

/dev/loop0: [0030]:33006 (/tmp/10G)

[root@armstrong tmp]#

3、在上述loop设备上创建pv、vg

[plain] view
plain copy







[root@armstrong tmp]# pvcreate qixiaozhen /dev/loop0

Device qixiaozhen not found (or ignored by filtering).

Physical volume "/dev/loop0" successfully created

[root@armstrong tmp]#

[root@armstrong tmp]# vgcreate qixiaozhen /dev/loop0

Volume group "qixiaozhen" successfully created

[root@armstrong tmp]#

4、创建用以测试用的lv

[plain] view
plain copy







[root@armstrong tmp]# lvcreate -n test001 -L 100M qixiaozhen

Logical volume "test001" created

[root@armstrong tmp]#

5、使用cryptsetup对上述lv进行加密（注意此处需要用户输入密码）

[plain] view
plain copy







[root@armstrong tmp]#

[root@armstrong tmp]# cryptsetup luksFormat /dev/qixiaozhen/test001

WARNING!

========

This will overwrite data on /dev/qixiaozhen/test001 irrevocably.

Are you sure? (Type uppercase yes): YES

Enter LUKS passphrase:

Verify passphrase:

[root@armstrong tmp]#

6、打开加密后的磁盘，生成供虚拟机使用磁盘（需要输入密码）

[plain] view
plain copy







[root@armstrong tmp]#

[root@armstrong tmp]# cryptsetup luksOpen /dev/qixiaozhen/test001 qixiaozhen_device

Enter passphrase for /dev/qixiaozhen/test001:

[root@armstrong tmp]#

7、在/dev/mapper路径下可以找到上述生成的qixiaozhen_device设备

[plain] view
plain copy







[root@armstrong tmp]#

[root@armstrong tmp]# ll /dev/mapper/qixiaozhen_device

lrwxrwxrwx. 1 root root 7 12月 3 23:48 /dev/mapper/qixiaozhen_device -> ../dm-4

[root@armstrong tmp]#

[root@armstrong tmp]#

8、分析逻辑卷test001与qixiaozhen_device设备间的关系

[plain] view
plain copy







[root@armstrong tmp]#

[root@armstrong tmp]# dmsetup table

fedora_armstrong-swap: 0 12255232 linear 8:9 2048

fedora_armstrong-root: 0 104857600 linear 8:9 995198976

qixiaozhen-test001: 0 204800 linear 7:0 2048

qixiaozhen_device: 0 200704 crypt aes-cbc-essiv:sha256 0000000000000000000000000000000000000000000000000000000000000000 0 253:3 4096

fedora_armstrong-home: 0 982941696 linear 8:9 12257280

[root@armstrong tmp]#

可见qixiaozhen_device设备是由test001设备偏移4096个扇区（2MB）而生成。

9、往qixiaozhen_device中写入部分数据，本文中通过格式化成ext4文件系统进行测试

[plain] view
plain copy







[root@armstrong tmp]# mkfs.ext4 /dev/mapper/qixiaozhen_device

mke2fs 1.42.5 (29-Jul-2012)

文件系统标签=

OS type: Linux

块大小=1024 (log=0)

分块大小=1024 (log=0)

Stride=0 blocks, Stripe width=0 blocks

25168 inodes, 100352 blocks

5017 blocks (5.00%) reserved for the super user

第一个数据块=1

Maximum filesystem blocks=67371008

13 block groups

8192 blocks per group, 8192 fragments per group

1936 inodes per group

Superblock backups stored on blocks:

8193, 24577, 40961, 57345, 73729

Allocating group tables: 完成

正在写入inode表: 完成

Creating journal (4096 blocks): 完成

Writing superblocks and filesystem accounting information: 完成

[root@armstrong tmp]#

10、下面分析使用dd命令读取test001偏移2MB和qixiaozhen_device进行比较。

[plain] view
plain copy







[root@armstrong tmp]# dd if=/dev/qixiaozhen/test001 bs=1M count=1 iflag=direct skip=2 | hexdump -C | less

00000000 4c a7 db 72 01 e4 18 1e bc 1e b9 bf fa 3b 25 01 |L..r.........;%.|

00000010 cc 22 46 96 aa b9 e8 2c 53 60 7c b3 61 4a 02 33 |."F....,S`|.aJ.3|

00000020 5d 39 4b 4f 6d b9 5a 57 16 fb a5 f9 c8 7f 9d 65 |]9KOm.ZW.......e|

00000030 3d 25 38 f5 4b e2 b0 6e ef 92 24 bd fc cf 56 07 |=%8.K..n..$...V.|

00000040 4b 5f d9 85 ad f4 01 5d dd a3 94 db 8b 58 0f a0 |K_.....].....X..|

00000050 03 0f 08 ad 7a 66 a8 3a 72 16 47 58 0c ba f9 d0 |....zf.:r.GX....|

00000060 60 34 b4 ba 69 32 27 8c f8 97 cd 58 86 3e ce 34 |`4..i2'....X.>.4|

00000070 ae 28 4f e1 c8 a9 90 5c 08 f9 30 9f a7 4b 6b 21 |.(O....\..0..Kk!|

00000080 ba d0 39 45 7a 0d a4 c5 a6 c9 a8 40 a2 56 c9 27 |..9Ez......@.V.'|

00000090 9d d4 77 05 77 ae b0 2a eb fa 3a 1d 1b d0 19 9d |..w.w..*..:.....|

000000a0 fe c3 d9 1c ca 3c 51 fa 84 d7 d8 a8 8d 42 1f e0 |.....<Q......B..|

000000b0 bd b0 09 84 69 25 6d 70 9f b6 78 25 87 52 07 70 |....i%mp..x%.R.p|

000000c0 e1 9c d9 9c 42 2d 1d fb 92 fd 9d 62 d3 58 27 79 |....B-.....b.X'y|

000000d0 2c b3 a5 a8 67 7d 3e 5d 01 15 3d ac 7e 44 77 67 |,...g}>]..=.~Dwg|

000000e0 8f 4f 71 e9 4e 63 06 4d 9a 78 7c 0f 60 4a da 3b |.Oq.Nc.M.x|.`J.;|

000000f0 0d f2 46 ac c0 c7 44 0c 61 dc 6c ef 00 c0 7b 9f |..F...D.a.l...{.|

00000100 60 55 94 7f 1f c2 87 1b 10 93 c0 78 92 6a 8f bf |`U.........x.j..|

00000110 6d 08 b1 b7 51 86 6e 83 46 26 cb fe 0d 0f c4 72 |m...Q.n.F&.....r|

00000120 b1 48 88 c5 05 be a9 f1 8a ac a2 ed f6 73 40 f5 |.H...........s@.|

00000130 74 51 76 ba f4 db 8c be f8 9d 73 14 e8 7b 8c 0f |tQv.......s..{..|

00000140 06 bb 89 f9 e4 2e 81 8e 5f c8 f4 a2 70 27 03 a5 |........_...p'..|

00000150 3e 93 80 b1 38 4d ad 12 9f ca 4c e8 01 5b 26 41 |>...8M....L..[&A|

00000160 73 f9 3f db 75 c5 d5 d9 c7 25 b9 6d 03 4d 39 98 |s.?.u....%.m.M9.|

00000170 72 7e 5f 10 31 be e9 92 fa 5c d5 54 9f 61 65 00 |r~_.1....\.T.ae.|

00000180 bd e0 4a 8f 8a 84 b4 41 5d b5 6c 42 f5 d5 ff 7a |..J....A].lB...z|

00000190 7f 97 c5 dc 73 74 c8 94 f7 03 f0 bd 8d 1d 69 9e |....st........i.|

000001a0 93 c4 52 52 d8 9e 21 6a 74 2d 1e d4 29 1e 1e 92 |..RR..!jt-..)...|

000001b0 8a 61 03 9d ed 0c ca c0 e5 33 38 49 d3 7e 5f 94 |.a.......38I.~_.|

000001c0 44 f9 ae 5d 5f 9c 5a 51 02 20 77 46 9c 46 23 6b |D..]_.ZQ. wF.F#k|

000001d0 8b 5e 15 28 4f 09 ce 3e 36 e8 62 21 e4 08 43 e9 |.^.(O..>6.b!..C.|

000001e0 b3 53 90 e2 68 fb 28 4d fa 24 e8 7a 3a ce b1 0a |.S..h.(M.$.z:...|

000001f0 f8 f1 ae a0 c9 57 82 9d 0e 56 6e f8 d9 d5 81 f8 |.....W...Vn.....|

00000200 9d f6 fa 8b d2 f2 bd cf d3 f0 ff e8 52 1f d1 f3 |............R...|

00000210 c1 aa ed db 78 55 f8 e3 7c 85 bf ec d2 a3 15 ec |....xU..|.......|

00000220 8b 6d 04 cf 2d 14 37 1b f4 22 06 5f c1 8d eb df |.m..-.7.."._....|

00000230 77 7b b9 f2 36 f7 b2 63 ad 23 06 ff 23 a1 b5 dc |w{..6..c.#..#...|

00000240 56 67 85 1f a2 f5 59 4c cc a0 c0 97 c1 b1 7e b2 |Vg....YL......~.|

00000250 e5 7f 6e 3c d2 ee 65 fa 52 10 ea b8 fb e0 38 e8 |..n<..e.R.....8.|

00000260 cf 95 9e 80 72 e4 5b 22 2b 9f 72 f6 22 2f e1 f7 |....r.["+.r."/..|

00000270 ec f4 49 cf c3 d2 f4 34 de 00 55 17 e9 1c cf 5a |..I....4..U....Z|

00000280 19 7f a0 f4 48 14 f5 ee 2f 75 76 ca 0e 94 9f 16 |....H.../uv.....|

00000290 51 aa 80 03 a9 e6 2c 69 31 d4 35 d5 d5 b7 e9 58 |Q.....,i1.5....X|

000002a0 bc a1 f5 8f 56 bc 12 94 9e d7 d8 cc 7d f7 61 bd |....V.......}.a.|

[plain] view
plain copy







[root@armstrong tmp]# dd if=/dev/mapper/qixiaozhen_device bs=1M count=1 iflag=direct | hexdump -C | less

00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|

*

00000400 50 62 00 00 00 88 01 00 99 13 00 00 81 65 01 00 |Pb...........e..|

00000410 45 62 00 00 01 00 00 00 00 00 00 00 00 00 00 00 |Eb..............|

00000420 00 20 00 00 00 20 00 00 90 07 00 00 00 00 00 00 |. ... ..........|

00000430 8c ff 9d 52 00 00 ff ff 53 ef 01 00 01 00 00 00 |...R....S.......|

00000440 8c ff 9d 52 00 00 00 00 00 00 00 00 01 00 00 00 |...R............|

00000450 00 00 00 00 0b 00 00 00 80 00 00 00 3c 00 00 00 |............<...|

00000460 42 02 00 00 79 00 00 00 0e 6c 42 94 c7 e6 4b 18 |B...y....lB...K.|

00000470 99 55 8d 81 b6 2c 5a 9a 00 00 00 00 00 00 00 00 |.U...,Z.........|

00000480 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|

*

000004c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................|

000004d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|

000004e0 08 00 00 00 00 00 00 00 00 00 00 00 25 2a 68 6f |............%*ho|

000004f0 14 38 4a d7 b2 35 fb 62 20 5c ec 36 01 01 00 00 |.8J..5.b \.6....|

00000500 0c 00 00 00 00 00 00 00 8c ff 9d 52 0a f3 01 00 |...........R....|

00000510 04 00 00 00 00 00 00 00 00 00 00 00 00 10 00 00 |................|

00000520 01 c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|

00000530 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|

00000540 00 00 00 00 00 00 00 00 00 00 00 00 00 00 40 00 |..............@.|

00000550 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|

00000560 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|

00000570 00 00 00 00 04 00 00 00 5f 11 00 00 00 00 00 00 |........_.......|

00000580 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|

*

00000800 03 01 00 00 13 01 00 00 23 01 00 00 8c 12 85 07 |........#.......|

00000810 02 00 00 00 00 00 00 00 00 00 00 00 85 07 b9 cc |................|

00000820 04 01 00 00 14 01 00 00 15 02 00 00 fe 1e 90 07 |................|

00000830 00 00 01 00 00 00 00 00 00 00 00 00 90 07 f4 ce |................|

00000840 05 01 00 00 15 01 00 00 07 03 00 00 00 20 90 07 |............. ..|

00000850 00 00 03 00 00 00 00 00 00 00 00 00 90 07 77 60 |..............w`|

00000860 06 01 00 00 16 01 00 00 f9 03 00 00 fe 1e 90 07 |................|

00000870 00 00 01 00 00 00 00 00 00 00 00 00 90 07 36 89 |..............6.|

00000880 07 01 00 00 17 01 00 00 eb 04 00 00 00 20 90 07 |............. ..|

00000890 00 00 03 00 00 00 00 00 00 00 00 00 90 07 d2 35 |...............5|

000008a0 08 01 00 00 18 01 00 00 dd 05 00 00 fe 1e 90 07 |................|

000008b0 00 00 01 00 00 00 00 00 00 00 00 00 90 07 e0 b7 |................|

000008c0 09 01 00 00 19 01 00 00 cf 06 00 00 00 10 90 07 |................|

000008d0 00 00 01 00 00 00 00 00 00 00 00 00 90 07 ad 33 |...............3|

000008e0 0a 01 00 00 1a 01 00 00 c1 07 00 00 fe 1e 90 07 |................|

000008f0 00 00 01 00 00 00 00 00 00 00 00 00 90 07 ee 88 |................|

00000900 0b 01 00 00 1b 01 00 00 b3 08 00 00 00 20 90 07 |............. ..|

结论：可见，直接在主机上读取裸设备上的裸数据时，从加密前设备和加密后设备上的是完全不同的。使用磁盘加密的方法，在删除磁盘时，不需要对磁盘进行格式化，直接把加密用的密码删除即可，再也不用往裸设备中填0实现格式化。