zram lz4 和 lzo 算法性能评估比较

本次评估是基于高通660 平台，Arm 64 位处理器，其中有 8 个核，4个小核 1.8G 频率，4个大核 2.2G 频率 ，主要对比 zram 的两个主流的压缩算法的性能差别。由于前段时间在看工作中发现这个平台默认只使用 lzo，但在社区上早就有了 lz4 的算法，且性能要比 lzo 算法在读方面要好 3 倍，于是就搜了一下 lz4 的压缩算法在 arm 平台上的实际表现，可惜资料很少，并且还有一些评估报告说 lz4 没有达到预期那么好，所以我确定自己着手实践一下，到底为什么这个平台默认只开 lzo 算法。

使能 LZ4

需要打开以下几个内核配置项

+CONFIG_ZRAM_LZ4_COMPRESS=y
+CONFIG_LZ4_DECOMPRESS=y
+CONFIG_LZ4_COMPRESS=y

重新编译后启动系统，可以通过往 sys/block/zram0/comp_algorithm 节点上写 lz4 来切换zram 的压缩算法

注意: 需要先 reset zram 设备后才能切换压缩算法，正确的切换方法为

echo 1 > /sys/class/block/zram0/reset
echo lz4 > sys/block/zram0/comp_algorithm

测试方法

由于 zram 被设计为 block 设备，所以可以通过块设备读写来评估 zram 的性能，在不挂载任何文件系统的情况下测试。

为了确保去运算能力的一致性，把 cpu 的频率都锁定到最高点测试。

由于sdm660 没有动态 cpu hotplug 的功能，因此 8 核全部在线测试。

分别测试单线程和多线程（8个）读写两种情况下的 lz4 和 lzo 性能。

以下是初始化命令。

swapoff /dev/block/zram0

echo 1 > /sys/class/block/zram0/reset

echo lzo/lz4 > /sys/class/block/zram0/comp_algorithm

echo 8 > /sys/class/block/zram0/max_comp_streams   # 设置最大压缩流为 8 个

echo 1610612736 > /sys/class/block/zram0/disksize    # 设置块设备大小为1.5G

echo 1843200 > /sys/devices/system/cpu/cpufreq/policy0/scaling_max_freq

echo 1843200 > /sys/devices/system/cpu/cpufreq/policy0/scaling_min_freq

echo 2208000 > /sys/devices/system/cpu/cpufreq/policy4/scaling_max_freq

echo 2208000 > /sys/devices/system/cpu/cpufreq/policy4/scaling_min_freq

然后，通过 fio 工具进行 io 测试，总 io 量为 500M，单次 io 的大小为 32 M。

fio -filename=/dev/block/zram0 -thread -rw=write -bs=32468k -size=500M -group_reporting -numjobs=1 -name=mytest

-rw :指定读/写

-numjobs :指定线程数

LZO 测试结果

单线程

写:

第一次测试结果:

mytest: (g=0): rw=write, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
fio-2.2.6
Starting 1 thread
Jobs: 1 (f=1)
mytest: (groupid=0, jobs=1): err= 0: pid=5149: Thu Dec 28 16:32:03 2017
write: io=519488KB, bw=660926KB/s, iops=20, runt=   786msec
clat (msec): min=17, max=81, avg=46.84, stdev=28.01
lat (msec): min=20, max=84, avg=49.17, stdev=27.91
clat percentiles (usec):
|  1.00th=[17536],  5.00th=[17536], 10.00th=[17792], 20.00th=[17792],
| 30.00th=[18304], 40.00th=[18816], 50.00th=[43264], 60.00th=[71168],
| 70.00th=[74240], 80.00th=[74240], 90.00th=[81408], 95.00th=[81408],
| 99.00th=[81408], 99.50th=[81408], 99.90th=[81408], 99.95th=[81408],
| 99.99th=[81408]
bw (KB  /s): min=777318, max=777318, per=100.00%, avg=777318.00, stdev= 0.00
lat (msec) : 20=43.75%, 50=12.50%, 100=43.75%
cpu          : usr=6.35%, sys=60.99%, ctx=106, majf=0, minf=3
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=0/w=16/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
WRITE: io=519488KB, aggrb=660926KB/s, minb=660926KB/s, maxb=660926KB/s, mint=786msec, maxt=786msec

Disk stats (read/write):
zram0: ios=0/53289, merge=0/0, ticks=0/360, in_queue=360, util=48.78%

第二次测试结果：

mytest: (g=0): rw=write, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
fio-2.2.6
Starting 1 thread
Jobs: 1 (f=1): [W(1)] [-.-% done] [0KB/0KB/0KB /s] [0/0/0 iops] [eta 00m:00s]
mytest: (groupid=0, jobs=1): err= 0: pid=5212: Thu Dec 28 16:34:49 2017
write: io=519488KB, bw=619914KB/s, iops=19, runt=   838msec
clat (msec): min=17, max=89, avg=50.01, stdev=31.28
lat (msec): min=19, max=92, avg=52.40, stdev=31.06
clat percentiles (usec):
|  1.00th=[17024],  5.00th=[17024], 10.00th=[17280], 20.00th=[17536],
| 30.00th=[17536], 40.00th=[18048], 50.00th=[43776], 60.00th=[72192],
| 70.00th=[79360], 80.00th=[79360], 90.00th=[87552], 95.00th=[89600],
| 99.00th=[89600], 99.50th=[89600], 99.90th=[89600], 99.95th=[89600],
| 99.99th=[89600]
bw (KB  /s): min=728987, max=728987, per=100.00%, avg=728987.00, stdev= 0.00
lat (msec) : 20=43.75%, 50=6.25%, 100=50.00%
cpu          : usr=21.48%, sys=41.77%, ctx=71, majf=0, minf=3
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=0/w=16/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
WRITE: io=519488KB, aggrb=619914KB/s, minb=619914KB/s, maxb=619914KB/s, mint=838msec, maxt=838msec

Disk stats (read/write):
zram0: ios=0/50866, merge=0/0, ticks=0/400, in_queue=400, util=53.98%

读:

第一次测试结果：

mytest: (g=0): rw=read, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
fio-2.2.6
Starting 1 thread

mytest: (groupid=0, jobs=1): err= 0: pid=5139: Thu Dec 28 16:49:24 2017
read : io=519488KB, bw=628160KB/s, iops=19, runt=   827msec
clat (msec): min=36, max=78, avg=51.71, stdev= 9.23
lat (msec): min=36, max=78, avg=51.71, stdev= 9.23
clat percentiles (usec):
|  1.00th=[36608],  5.00th=[36608], 10.00th=[39168], 20.00th=[47360],
| 30.00th=[49920], 40.00th=[51968], 50.00th=[52480], 60.00th=[52992],
| 70.00th=[54016], 80.00th=[55040], 90.00th=[56064], 95.00th=[78336],
| 99.00th=[78336], 99.50th=[78336], 99.90th=[78336], 99.95th=[78336],
| 99.99th=[78336]
bw (KB  /s): min=617262, max=617262, per=98.27%, avg=617262.00, stdev= 0.00
lat (msec) : 50=31.25%, 100=68.75%
cpu          : usr=0.00%, sys=97.94%, ctx=14, majf=0, minf=8121
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=16/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
READ: io=519488KB, aggrb=628159KB/s, minb=628159KB/s, maxb=628159KB/s, mint=827msec, maxt=827msec

Disk stats (read/write):
zram0: ios=97677/0, merge=0/0, ticks=330/0, in_queue=340, util=45.88%

第二次测试结果：

mytest: (g=0): rw=read, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
fio-2.2.6
Starting 1 thread

mytest: (groupid=0, jobs=1): err= 0: pid=5204: Thu Dec 28 16:34:16 2017
read : io=519488KB, bw=698237KB/s, iops=21, runt=   744msec
clat (msec): min=36, max=71, avg=46.52, stdev=10.11
lat (msec): min=36, max=71, avg=46.52, stdev=10.11
clat percentiles (usec):
|  1.00th=[37120],  5.00th=[37120], 10.00th=[38144], 20.00th=[38656],
| 30.00th=[38656], 40.00th=[39168], 50.00th=[39680], 60.00th=[52992],
| 70.00th=[54016], 80.00th=[54016], 90.00th=[55552], 95.00th=[71168],
| 99.00th=[71168], 99.50th=[71168], 99.90th=[71168], 99.95th=[71168],
| 99.99th=[71168]
bw (KB  /s): min=737909, max=737909, per=100.00%, avg=737909.00, stdev= 0.00
lat (msec) : 50=56.25%, 100=43.75%
cpu          : usr=0.00%, sys=98.12%, ctx=10, majf=0, minf=8121
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=16/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
READ: io=519488KB, aggrb=698236KB/s, minb=698236KB/s, maxb=698236KB/s, mint=744msec, maxt=744msec

Disk stats (read/write):
zram0: ios=114357/0, merge=0/0, ticks=410/0, in_queue=410, util=55.11%

压缩率

sdm660:/ # cat /sys/block/zram0/mm_stat
531955712 41390528 43917312        0 43917312        0        0

8 线程

写:

第一次测试结果:

mytest: (g=0): rw=write, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
...
fio-2.2.6
Starting 8 threads
Jobs: 1 (f=1): [_(3),W(1),_(4)] [-.-% done] [0KB/1802MB/0KB /s] [0/56/0 iops] [eta 00m:00s]
mytest: (groupid=0, jobs=8): err= 0: pid=5574: Thu Dec 28 17:31:41 2017
write: io=4058.6MB, bw=3008.6MB/s, iops=94, runt=  1349msec
clat (msec): min=11, max=143, avg=65.31, stdev=26.37
lat (msec): min=13, max=165, avg=72.88, stdev=28.55
clat percentiles (msec):
|  1.00th=[   18],  5.00th=[   26], 10.00th=[   29], 20.00th=[   48],
| 30.00th=[   52], 40.00th=[   56], 50.00th=[   64], 60.00th=[   71],
| 70.00th=[   76], 80.00th=[   86], 90.00th=[  103], 95.00th=[  111],
| 99.00th=[  139], 99.50th=[  143], 99.90th=[  143], 99.95th=[  143],
| 99.99th=[  143]
bw (KB  /s): min=173625, max=559793, per=12.47%, avg=384234.75, stdev=113215.08
lat (msec) : 20=1.56%, 50=26.56%, 100=60.94%, 250=10.94%
cpu          : usr=9.32%, sys=72.94%, ctx=10109, majf=0, minf=3
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=0/w=128/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
WRITE: io=4058.6MB, aggrb=3008.6MB/s, minb=3008.6MB/s, maxb=3008.6MB/s, mint=1349msec, maxt=1349msec

Disk stats (read/write):
zram0: ios=0/60263, merge=0/0, ticks=0/900, in_queue=900, util=60.32%

第二次测试结果：

mytest: (g=0): rw=write, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
...
fio-2.2.6
Starting 8 threads
Jobs: 1 (f=1): [W(1),_(7)] [-.-% done] [0KB/1614MB/0KB /s] [0/50/0 iops] [eta 00m:00s]
mytest: (groupid=0, jobs=8): err= 0: pid=5600: Thu Dec 28 17:32:15 2017
write: io=4058.6MB, bw=3146.2MB/s, iops=99, runt=  1290msec
clat (msec): min=25, max=166, avg=68.87, stdev=16.38
lat (msec): min=34, max=202, avg=78.04, stdev=18.45
clat percentiles (msec):
|  1.00th=[   32],  5.00th=[   48], 10.00th=[   52], 20.00th=[   57],
| 30.00th=[   62], 40.00th=[   68], 50.00th=[   70], 60.00th=[   73],
| 70.00th=[   74], 80.00th=[   78], 90.00th=[   85], 95.00th=[   93],
| 99.00th=[  103], 99.50th=[  167], 99.90th=[  167], 99.95th=[  167],
| 99.99th=[  167]
bw (KB  /s): min=314003, max=458911, per=12.45%, avg=401197.88, stdev=36107.38
lat (msec) : 50=7.81%, 100=89.06%, 250=3.12%
cpu          : usr=11.15%, sys=74.17%, ctx=2878, majf=0, minf=5
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=0/w=128/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
WRITE: io=4058.6MB, aggrb=3146.2MB/s, minb=3146.2MB/s, maxb=3146.2MB/s, mint=1290msec, maxt=1290msec

Disk stats (read/write):
zram0: ios=0/36094, merge=0/0, ticks=0/610, in_queue=610, util=49.15%

读:

第一次测试结果：

mytest: (g=0): rw=read, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
...
fio-2.2.6
Starting 8 threads
Jobs: 8 (f=8)
mytest: (groupid=0, jobs=8): err= 0: pid=5379: Thu Dec 28 17:10:16 2017
read : io=4058.6MB, bw=3986.8MB/s, iops=125, runt=  1018msec
clat (msec): min=34, max=138, avg=63.37, stdev=13.64
lat (msec): min=34, max=138, avg=63.38, stdev=13.64
clat percentiles (msec):
|  1.00th=[   40],  5.00th=[   55], 10.00th=[   56], 20.00th=[   59],
| 30.00th=[   60], 40.00th=[   60], 50.00th=[   61], 60.00th=[   61],
| 70.00th=[   62], 80.00th=[   65], 90.00th=[   69], 95.00th=[   87],
| 99.00th=[  135], 99.50th=[  139], 99.90th=[  139], 99.95th=[  139],
| 99.99th=[  139]
bw (KB  /s): min=454552, max=566302, per=12.05%, avg=491748.33, stdev=29880.69
lat (msec) : 50=1.56%, 100=93.75%, 250=4.69%
cpu          : usr=0.00%, sys=85.58%, ctx=5283, majf=0, minf=64942
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=128/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
READ: io=4058.6MB, aggrb=3986.8MB/s, minb=3986.8MB/s, maxb=3986.8MB/s, mint=1018msec, maxt=1018msec

Disk stats (read/write):
zram0: ios=112713/0, merge=0/0, ticks=890/0, in_queue=910, util=57.52%

第二次测试结果：

mytest: (g=0): rw=read, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
...
fio-2.2.6
Starting 8 threads
Jobs: 8 (f=8)
mytest: (groupid=0, jobs=8): err= 0: pid=5562: Thu Dec 28 17:31:20 2017
read : io=4058.6MB, bw=3975.3MB/s, iops=125, runt=  1021msec
clat (msec): min=49, max=132, avg=63.26, stdev=12.85
lat (msec): min=49, max=132, avg=63.26, stdev=12.85
clat percentiles (msec):
|  1.00th=[   50],  5.00th=[   54], 10.00th=[   55], 20.00th=[   57],
| 30.00th=[   60], 40.00th=[   61], 50.00th=[   61], 60.00th=[   62],
| 70.00th=[   62], 80.00th=[   65], 90.00th=[   72], 95.00th=[   87],
| 99.00th=[  127], 99.50th=[  133], 99.90th=[  133], 99.95th=[  133],
| 99.99th=[  133]
bw (KB  /s): min=475721, max=488240, per=11.92%, avg=485303.88, stdev=3954.66
lat (msec) : 50=0.78%, 100=95.31%, 250=3.91%
cpu          : usr=0.37%, sys=84.56%, ctx=4863, majf=0, minf=64942
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=128/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
READ: io=4058.6MB, aggrb=3975.3MB/s, minb=3975.3MB/s, maxb=3975.3MB/s, mint=1021msec, maxt=1021msec

Disk stats (read/write):
zram0: ios=113421/0, merge=0/0, ticks=840/0, in_queue=900, util=57.46%

mytest: (g=0): rw=read, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1

压缩率

sdm660:/ # cat sys/block/zram0/mm_stat
531955712 42136643 47136768        0 49504256        0        0

LZ4 测试结果

单线程

写：

第一次测试结果：

mytest: (g=0): rw=write, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
fio-2.2.6
Starting 1 thread
Jobs: 1 (f=1)
mytest: (groupid=0, jobs=1): err= 0: pid=5252: Thu Dec 28 16:35:49 2017
write: io=519488KB, bw=829853KB/s, iops=25, runt=   626msec
clat (msec): min=17, max=67, avg=36.56, stdev=19.18
lat (msec): min=19, max=69, avg=39.15, stdev=19.10
clat percentiles (usec):
|  1.00th=[17536],  5.00th=[17536], 10.00th=[18048], 20.00th=[18304],
| 30.00th=[19072], 40.00th=[19584], 50.00th=[21632], 60.00th=[43776],
| 70.00th=[55552], 80.00th=[56064], 90.00th=[62208], 95.00th=[67072],
| 99.00th=[67072], 99.50th=[67072], 99.90th=[67072], 99.95th=[67072],
| 99.99th=[67072]
bw (KB  /s): min=875935, max=875935, per=100.00%, avg=875935.00, stdev= 0.00
lat (msec) : 20=43.75%, 50=25.00%, 100=31.25%
cpu          : usr=7.99%, sys=73.48%, ctx=71, majf=0, minf=3
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=0/w=16/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
WRITE: io=519488KB, aggrb=829853KB/s, minb=829853KB/s, maxb=829853KB/s, mint=626msec, maxt=626msec

Disk stats (read/write):
zram0: ios=0/70534, merge=0/0, ticks=0/380, in_queue=380, util=51.28%

第二次测试结果：

mytest: (g=0): rw=write, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
fio-2.2.6
Starting 1 thread
Jobs: 1 (f=1)
mytest: (groupid=0, jobs=1): err= 0: pid=5172: Thu Dec 28 16:51:17 2017
write: io=519488KB, bw=825895KB/s, iops=25, runt=   629msec
clat (msec): min=17, max=63, avg=36.88, stdev=19.06
lat (msec): min=18, max=66, avg=39.31, stdev=19.28
clat percentiles (usec):
|  1.00th=[17280],  5.00th=[17280], 10.00th=[18048], 20.00th=[18048],
| 30.00th=[19328], 40.00th=[19584], 50.00th=[21632], 60.00th=[47360],
| 70.00th=[56576], 80.00th=[57088], 90.00th=[59136], 95.00th=[63232],
| 99.00th=[63232], 99.50th=[63232], 99.90th=[63232], 99.95th=[63232],
| 99.99th=[63232]
bw (KB  /s): min=907289, max=907289, per=100.00%, avg=907289.00, stdev= 0.00
lat (msec) : 20=43.75%, 50=18.75%, 100=37.50%
cpu          : usr=6.36%, sys=74.72%, ctx=53, majf=0, minf=3
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=0/w=16/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
WRITE: io=519488KB, aggrb=825895KB/s, minb=825895KB/s, maxb=825895KB/s, mint=629msec, maxt=629msec

Disk stats (read/write):
zram0: ios=0/70067, merge=0/0, ticks=0/420, in_queue=430, util=58.03%

读：

第一次测试结果：

mytest: (g=0): rw=read, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
fio-2.2.6
Starting 1 thread

mytest: (groupid=0, jobs=1): err= 0: pid=5285: Thu Dec 28 16:37:29 2017
read : io=519488KB, bw=733740KB/s, iops=22, runt=   708msec
clat (msec): min=34, max=69, avg=44.24, stdev= 8.35
lat (msec): min=34, max=69, avg=44.24, stdev= 8.35
clat percentiles (usec):
|  1.00th=[34560],  5.00th=[34560], 10.00th=[35072], 20.00th=[35584],
| 30.00th=[42240], 40.00th=[43776], 50.00th=[43776], 60.00th=[44800],
| 70.00th=[46336], 80.00th=[46848], 90.00th=[50432], 95.00th=[70144],
| 99.00th=[70144], 99.50th=[70144], 99.90th=[70144], 99.95th=[70144],
| 99.99th=[70144]
bw (KB  /s): min=753609, max=753609, per=100.00%, avg=753609.00, stdev= 0.00
lat (msec) : 50=87.50%, 100=12.50%
cpu          : usr=0.00%, sys=97.46%, ctx=10, majf=0, minf=8121
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=16/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
READ: io=519488KB, aggrb=733740KB/s, minb=733740KB/s, maxb=733740KB/s, mint=708msec, maxt=708msec

Disk stats (read/write):
zram0: ios=118616/0, merge=0/0, ticks=340/0, in_queue=340, util=45.76%

第二次测试结果：

mytest: (g=0): rw=read, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
fio-2.2.6
Starting 1 thread

mytest: (groupid=0, jobs=1): err= 0: pid=5230: Thu Dec 28 16:56:36 2017
read : io=519488KB, bw=751792KB/s, iops=23, runt=   691msec
clat (msec): min=34, max=74, avg=43.22, stdev= 9.95
lat (msec): min=34, max=74, avg=43.22, stdev= 9.95
clat percentiles (usec):
|  1.00th=[34048],  5.00th=[34048], 10.00th=[34048], 20.00th=[34560],
| 30.00th=[35072], 40.00th=[43264], 50.00th=[43776], 60.00th=[45312],
| 70.00th=[45824], 80.00th=[46336], 90.00th=[47872], 95.00th=[74240],
| 99.00th=[74240], 99.50th=[74240], 99.90th=[74240], 99.95th=[74240],
| 99.99th=[74240]
bw (KB  /s): min=771516, max=771516, per=100.00%, avg=771516.00, stdev= 0.00
lat (msec) : 50=93.75%, 100=6.25%
cpu          : usr=0.00%, sys=98.41%, ctx=13, majf=0, minf=8121
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=16/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
READ: io=519488KB, aggrb=751791KB/s, minb=751791KB/s, maxb=751791KB/s, mint=691msec, maxt=691msec

Disk stats (read/write):
zram0: ios=120736/0, merge=0/0, ticks=350/0, in_queue=350, util=47.17%

压缩率

sdm660:/ # cat sys/block/zram0/mm_stat
531955712 41000087 43372544        0 51195904        0        0

8 线程

写：

第一次测试结果：

mytest: (g=0): rw=write, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
...
fio-2.2.6
Starting 8 threads
Jobs: 1 (f=1): [_(5),W(1),_(2)] [-.-% done] [0KB/1639MB/0KB /s] [0/51/0 iops] [eta 00m:00s]
mytest: (groupid=0, jobs=8): err= 0: pid=5647: Thu Dec 28 17:34:11 2017
write: io=4058.6MB, bw=3107.6MB/s, iops=98, runt=  1306msec
clat (msec): min=11, max=142, avg=68.53, stdev=16.41
lat (msec): min=14, max=149, avg=77.46, stdev=17.37
clat percentiles (msec):
|  1.00th=[   19],  5.00th=[   46], 10.00th=[   51], 20.00th=[   61],
| 30.00th=[   65], 40.00th=[   68], 50.00th=[   70], 60.00th=[   72],
| 70.00th=[   75], 80.00th=[   77], 90.00th=[   85], 95.00th=[   92],
| 99.00th=[  125], 99.50th=[  143], 99.90th=[  143], 99.95th=[  143],
| 99.99th=[  143]
bw (KB  /s): min=326311, max=456492, per=12.49%, avg=397464.25, stdev=36325.79
lat (msec) : 20=2.34%, 50=7.03%, 100=88.28%, 250=2.34%
cpu          : usr=9.94%, sys=74.54%, ctx=2923, majf=0, minf=3
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=0/w=128/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
WRITE: io=4058.6MB, aggrb=3107.6MB/s, minb=3107.6MB/s, maxb=3107.6MB/s, mint=1306msec, maxt=1306msec

Disk stats (read/write):
zram0: ios=0/68515, merge=0/0, ticks=0/620, in_queue=620, util=49.80%

第二次测试结果：

mytest: (g=0): rw=write, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
...
fio-2.2.6
Starting 8 threads
Jobs: 1 (f=1): [_(5),W(1),_(2)] [-.-% done] [0KB/1789MB/0KB /s] [0/56/0 iops] [eta 00m:00s]
mytest: (groupid=0, jobs=8): err= 0: pid=5659: Thu Dec 28 17:34:30 2017
write: io=4058.6MB, bw=3024.3MB/s, iops=95, runt=  1342msec
clat (msec): min=16, max=164, avg=65.75, stdev=22.80
lat (msec): min=20, max=166, avg=73.42, stdev=23.86
clat percentiles (msec):
|  1.00th=[   18],  5.00th=[   26], 10.00th=[   38], 20.00th=[   49],
| 30.00th=[   60], 40.00th=[   67], 50.00th=[   68], 60.00th=[   70],
| 70.00th=[   74], 80.00th=[   77], 90.00th=[   88], 95.00th=[  102],
| 99.00th=[  141], 99.50th=[  165], 99.90th=[  165], 99.95th=[  165],
| 99.99th=[  165]
bw (KB  /s): min=277504, max=510860, per=12.35%, avg=382469.31, stdev=72763.75
lat (msec) : 20=3.91%, 50=17.97%, 100=72.66%, 250=5.47%
cpu          : usr=9.40%, sys=74.13%, ctx=5562, majf=0, minf=3
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=0/w=128/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
WRITE: io=4058.6MB, aggrb=3024.3MB/s, minb=3024.3MB/s, maxb=3024.3MB/s, mint=1342msec, maxt=1342msec

Disk stats (read/write):
zram0: ios=0/85372, merge=0/0, ticks=0/920, in_queue=930, util=61.92%

读：

第一次测试结果：

mytest: (g=0): rw=read, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
...
fio-2.2.6
Starting 8 threads
Jobs: 8 (f=8)
mytest: (groupid=0, jobs=8): err= 0: pid=5686: Thu Dec 28 17:35:19 2017
read : io=4058.6MB, bw=3967.3MB/s, iops=125, runt=  1023msec
clat (msec): min=49, max=129, avg=63.64, stdev=13.01
lat (msec): min=49, max=129, avg=63.64, stdev=13.01
clat percentiles (msec):
|  1.00th=[   50],  5.00th=[   52], 10.00th=[   56], 20.00th=[   59],
| 30.00th=[   61], 40.00th=[   61], 50.00th=[   62], 60.00th=[   62],
| 70.00th=[   63], 80.00th=[   64], 90.00th=[   67], 95.00th=[   93],
| 99.00th=[  120], 99.50th=[  130], 99.90th=[  130], 99.95th=[  130],
| 99.99th=[  130]
bw (KB  /s): min=488240, max=492872, per=12.10%, avg=491479.50, stdev=1854.08
lat (msec) : 50=2.34%, 100=92.97%, 250=4.69%
cpu          : usr=0.24%, sys=82.04%, ctx=6451, majf=0, minf=64944
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=128/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
READ: io=4058.6MB, aggrb=3967.3MB/s, minb=3967.3MB/s, maxb=3967.3MB/s, mint=1023msec, maxt=1023msec

Disk stats (read/write):
zram0: ios=114639/0, merge=0/0, ticks=810/0, in_queue=850, util=53.48%

第二次测试结果：

mytest: (g=0): rw=read, bs=32468K-32468K/32468K-32468K/32468K-32468K, ioengine=sync, iodepth=1
...
fio-2.2.6
Starting 8 threads
Jobs: 8 (f=8)
mytest: (groupid=0, jobs=8): err= 0: pid=5708: Thu Dec 28 17:35:39 2017
read : io=4058.6MB, bw=4002.5MB/s, iops=126, runt=  1014msec
clat (msec): min=41, max=128, avg=63.14, stdev=11.99
lat (msec): min=41, max=128, avg=63.14, stdev=11.99
clat percentiles (msec):
|  1.00th=[   51],  5.00th=[   52], 10.00th=[   52], 20.00th=[   59],
| 30.00th=[   61], 40.00th=[   62], 50.00th=[   62], 60.00th=[   63],
| 70.00th=[   63], 80.00th=[   64], 90.00th=[   68], 95.00th=[   97],
| 99.00th=[  123], 99.50th=[  129], 99.90th=[  129], 99.95th=[  129],
| 99.99th=[  129]
bw (KB  /s): min=481007, max=506323, per=12.26%, avg=502299.00, stdev=8682.96
lat (msec) : 50=0.78%, 100=96.88%, 250=2.34%
cpu          : usr=0.86%, sys=82.86%, ctx=6763, majf=0, minf=64943
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=128/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
READ: io=4058.6MB, aggrb=4002.5MB/s, minb=4002.5MB/s, maxb=4002.5MB/s, mint=1014msec, maxt=1014msec

Disk stats (read/write):
zram0: ios=113066/0, merge=0/0, ticks=1010/0, in_queue=1030, util=63.57%

压缩率

sdm660:/ # cat sys/block/zram0/mm_stat
531955712 42835978 45449216        0 49844224        0        0

结论

测试	LZO	LZ4	delta ( lz4 - lz0 )
单线程写	640M/s	828M/s	188M/s (29%)
单线程读	663M/s	743M/s	80M/s (12%)
8 线程写	3077M/s	3065M/s	-12M/s
8 线程读	3980M/s	3984M/s	4M/s
压缩率	12.85倍	12.97倍	0.12倍

上表的压缩率的计算方法：末压缩的内存除以压缩后的内存占用。

上表对比的是两次测试结果的平均读写速度，从单线来看，lz4 的比 lz0 要快，写可快 29%，读可以快 12%，但压缩仍然可以达到 lzo 的水平。

从多线程来看，两者的速度差不多。

在读方面 lz4 只比 lzo 快了 12 %，这与社区所说的3倍速度是有差别的，可能这与测试的读写量有关，也或许是在整个读流程中，压缩算法只占一小部分时间，即使这部分时间快3倍，但放在整个读 io 过程中，这个时间优化的影响就被缩小了，或许增大每一笔 io 的请求量可突出这部分时间吧，大家有兴趣可以尝试一下。