文本摘要评测工具ROUGE的搭建和测试
2017-12-22 11:22
1256 查看
文本摘要任务中最常用的评价方法是ROUGE(Recall-Oriented
Understudy for Gisting Evaluation)。ROUGE受到了机器翻译自动评价方法BLEU的启发,不同之处在于,采用召回率来作为指标。基本思想是将模型生成的摘要与参考摘要的n元组贡献统计量作为评判依据。
现在主要采用软件是PERL语言编写的版本,见地址:
(https://github.com/andersjo/pyrouge/tree/master/tools/ROUGE-1.5.5)
然而这个工具的搭建,相对还比较麻烦,因此把整个搭建的过程记录一下:
(1)安装PERL语言,一般的Ubuntu环境都具备
(2)安装PERL语言的相关库,主要是XML语言解析器
(3)对数据进行处理,主要是WordNet数据的处理,主要是原来给的文件会存在无法打开的问题,即报如下错误:(Cannot open exception db file for reading: data/WordNet-2.0.exc.db)
处理步骤如下:
然后进行测试:
./ROUGE-1.5.5.pl -e data -c 95 -2 -1 -U -r 1000 -n 4 -w 1.2 -a ROUGE-test.xml
其中测试文件可以从如下网址下载:ROUGE-test.xml (https://raw.githubusercontent.com/summanlp/evaluation/master/ROUGE-RELEASE-1.5.5/sample-test/ROUGE-test.xml)
测试呈现的结果如下:
Understudy for Gisting Evaluation)。ROUGE受到了机器翻译自动评价方法BLEU的启发,不同之处在于,采用召回率来作为指标。基本思想是将模型生成的摘要与参考摘要的n元组贡献统计量作为评判依据。
现在主要采用软件是PERL语言编写的版本,见地址:
(https://github.com/andersjo/pyrouge/tree/master/tools/ROUGE-1.5.5)
然而这个工具的搭建,相对还比较麻烦,因此把整个搭建的过程记录一下:
(1)安装PERL语言,一般的Ubuntu环境都具备
(2)安装PERL语言的相关库,主要是XML语言解析器
(3)对数据进行处理,主要是WordNet数据的处理,主要是原来给的文件会存在无法打开的问题,即报如下错误:(Cannot open exception db file for reading: data/WordNet-2.0.exc.db)
处理步骤如下:
cd pythonrouge/RELEASE-1.5.5/data/ rm WordNet-2.0.exc.db ./WordNet-2.0-Exceptions/buildExeptionDB.pl ./WordNet-2.0-Exceptions ./smart_common_words.txt ./WordNet-2.0.exc.db
然后进行测试:
./ROUGE-1.5.5.pl -e data -c 95 -2 -1 -U -r 1000 -n 4 -w 1.2 -a ROUGE-test.xml
其中测试文件可以从如下网址下载:ROUGE-test.xml (https://raw.githubusercontent.com/summanlp/evaluation/master/ROUGE-RELEASE-1.5.5/sample-test/ROUGE-test.xml)
测试呈现的结果如下:
omnisky@omnisky:~/software/ROUGE-1.5.5$ ./ROUGE-1.5.5.pl -e data -c 95 -2 -1 -U -r 1000 -n 4 -w 1.2 -b 75 -m -a ROUGE-test.xml --------------------------------------------- 11 ROUGE-1 Average_R: 0.22536 (95%-conf.int. 0.18124 - 0.27016) 11 ROUGE-1 Average_P: 0.20359 (95%-conf.int. 0.17004 - 0.23980) 11 ROUGE-1 Average_F: 0.21027 (95%-conf.int. 0.17278 - 0.24804) --------------------------------------------- 11 ROUGE-2 Average_R: 0.03522 (95%-conf.int. 0.01812 - 0.05479) 11 ROUGE-2 Average_P: 0.02964 (95%-conf.int. 0.01698 - 0.04433) 11 ROUGE-2 Average_F: 0.03109 (95%-conf.int. 0.01669 - 0.04702) --------------------------------------------- 11 ROUGE-3 Average_R: 0.00243 (95%-conf.int. 0.00000 - 0.00774) 11 ROUGE-3 Average_P: 0.00171 (95%-conf.int. 0.00000 - 0.00545) 11 ROUGE-3 Average_F: 0.00201 (95%-conf.int. 0.00000 - 0.00640) --------------------------------------------- 11 ROUGE-4 Average_R: 0.00000 (95%-conf.int. 0.00000 - 0.00000) 11 ROUGE-4 Average_P: 0.00000 (95%-conf.int. 0.00000 - 0.00000) 11 ROUGE-4 Average_F: 0.00000 (95%-conf.int. 0.00000 - 0.00000) --------------------------------------------- 11 ROUGE-L Average_R: 0.17144 (95%-conf.int. 0.14160 - 0.19972) 11 ROUGE-L Average_P: 0.15459 (95%-conf.int. 0.13442 - 0.17638) 11 ROUGE-L Average_F: 0.15979 (95%-conf.int. 0.13524 - 0.18293) --------------------------------------------- 11 ROUGE-W-1.2 Average_R: 0.10366 (95%-conf.int. 0.08657 - 0.12032) 11 ROUGE-W-1.2 Average_P: 0.14874 (95%-conf.int. 0.13072 - 0.16891) 11 ROUGE-W-1.2 Average_F: 0.12004 (95%-conf.int. 0.10260 - 0.13592) --------------------------------------------- 11 ROUGE-S* Average_R: 0.02919 (95%-conf.int. 0.01857 - 0.04092) 11 ROUGE-S* Average_P: 0.02341 (95%-conf.int. 0.01557 - 0.03158) 11 ROUGE-S* Average_F: 0.02400 (95%-conf.int. 0.01604 - 0.03186) --------------------------------------------- 11 ROUGE-SU* Average_R: 0.06207 (95%-conf.int. 0.04583 - 0.07847) 11 ROUGE-SU* Average_P: 0.05303 (95%-conf.int. 0.04009 - 0.06763) 11 ROUGE-SU* Average_F: 0.05310 (95%-conf.int. 0.04099 - 0.06553) --------------------------------------------- 12 ROUGE-1 Average_R: 0.24238 (95%-conf.int. 0.18967 - 0.29513) 12 ROUGE-1 Average_P: 0.25533 (95%-conf.int. 0.19308 - 0.31825) 12 ROUGE-1 Average_F: 0.24485 (95%-conf.int. 0.19150 - 0.30122) --------------------------------------------- 12 ROUGE-2 Average_R: 0.05210 (95%-conf.int. 0.02453 - 0.08236) 12 ROUGE-2 Average_P: 0.05569 (95%-conf.int. 0.02581 - 0.08922) 12 ROUGE-2 Average_F: 0.05265 (95%-conf.int. 0.02501 - 0.08296) --------------------------------------------- 12 ROUGE-3 Average_R: 0.01023 (95%-conf.int. 0.00114 - 0.02271) 12 ROUGE-3 Average_P: 0.01027 (95%-conf.int. 0.00125 - 0.02146) 12 ROUGE-3 Average_F: 0.00995 (95%-conf.int. 0.00119 - 0.02145) --------------------------------------------- 12 ROUGE-4 Average_R: 0.00000 (95%-conf.int. 0.00000 - 0.00000) 12 ROUGE-4 Average_P: 0.00000 (95%-conf.int. 0.00000 - 0.00000) 12 ROUGE-4 Average_F: 0.00000 (95%-conf.int. 0.00000 - 0.00000) --------------------------------------------- 12 ROUGE-L Average_R: 0.18008 (95%-conf.int. 0.13709 - 0.22455) 12 ROUGE-L Average_P: 0.18728 (95%-conf.int. 0.14248 - 0.23337) 12 ROUGE-L Average_F: 0.18062 (95%-conf.int. 0.13810 - 0.22318) --------------------------------------------- 12 ROUGE-W-1.2 Average_R: 0.10847 (95%-conf.int. 0.08398 - 0.13339) 12 ROUGE-W-1.2 Average_P: 0.17875 (95%-conf.int. 0.13756 - 0.22048) 12 ROUGE-W-1.2 Average_F: 0.13289 (95%-conf.int. 0.10403 - 0.16220) --------------------------------------------- 12 ROUGE-S* Average_R: 0.03833 (95%-conf.int. 0.02085 - 0.05926) 12 ROUGE-S* Average_P: 0.04319 (95%-conf.int. 0.02107 - 0.06921) 12 ROUGE-S* Average_F: 0.03788 (95%-conf.int. 0.01997 - 0.05816) --------------------------------------------- 12 ROUGE-SU* Average_R: 0.07160 (95%-conf.int. 0.04882 - 0.09699) 12 ROUGE-SU* Average_P: 0.08071 (95%-conf.int. 0.05108 - 0.11638) 12 ROUGE-SU* Average_F: 0.07160 (95%-conf.int. 0.04794 - 0.09681) --------------------------------------------- 13 ROUGE-1 Average_R: 0.20161 (95%-conf.int. 0.15184 - 0.25908) 13 ROUGE-1 Average_P: 0.19956 (95%-conf.int. 0.14511 - 0.25978) 13 ROUGE-1 Average_F: 0.20030 (95%-conf.int. 0.14833 - 0.25923) --------------------------------------------- 13 ROUGE-2 Average_R: 0.04886 (95%-conf.int. 0.02609 - 0.07824) 13 ROUGE-2 Average_P: 0.04829 (95%-conf.int. 0.02445 - 0.07861) 13 ROUGE-2 Average_F: 0.04846 (95%-conf.int. 0.02523 - 0.07828) --------------------------------------------- 13 ROUGE-3 Average_R: 0.00887 (95%-conf.int. 0.00250 - 0.01758) 13 ROUGE-3 Average_P: 0.00909 (95%-conf.int. 0.00250 - 0.01804) 13 ROUGE-3 Average_F: 0.00897 (95%-conf.int. 0.00250 - 0.01758) --------------------------------------------- 13 ROUGE-4 Average_R: 0.00000 (95%-conf.int. 0.00000 - 0.00000) 13 ROUGE-4 Average_P: 0.00000 (95%-conf.int. 0.00000 - 0.00000) 13 ROUGE-4 Average_F: 0.00000 (95%-conf.int. 0.00000 - 0.00000) --------------------------------------------- 13 ROUGE-L Average_R: 0.17044 (95%-conf.int. 0.12873 - 0.21975) 13 ROUGE-L Average_P: 0.16849 (95%-conf.int. 0.12400 - 0.22144) 13 ROUGE-L Average_F: 0.16919 (95%-conf.int. 0.12604 - 0.21969) --------------------------------------------- 13 ROUGE-W-1.2 Average_R: 0.10327 (95%-conf.int. 0.08048 - 0.12969) 13 ROUGE-W-1.2 Average_P: 0.16067 (95%-conf.int. 0.12237 - 0.20421) 13 ROUGE-W-1.2 Average_F: 0.12550 (95%-conf.int. 0.09682 - 0.15816) --------------------------------------------- 13 ROUGE-S* Average_R: 0.03974 (95%-conf.int. 0.02107 - 0.06491) 13 ROUGE-S* Average_P: 0.04116 (95%-conf.int. 0.01983 - 0.07039) 13 ROUGE-S* Average_F: 0.04018 (95%-conf.int. 0.02016 - 0.06705) --------------------------------------------- 13 ROUGE-SU* Average_R: 0.06653 (95%-conf.int. 0.04305 - 0.09595) 13 ROUGE-SU* Average_P: 0.06719 (95%-conf.int. 0.04110 - 0.10081) 13 ROUGE-SU* Average_F: 0.06650 (95%-conf.int. 0.04165 - 0.09775) --------------------------------------------- 14 ROUGE-1 Average_R: 0.23816 (95%-conf.int. 0.18633 - 0.28642) 14 ROUGE-1 Average_P: 0.20187 (95%-conf.int. 0.15801 - 0.24672) 14 ROUGE-1 Average_F: 0.21741 (95%-conf.int. 0.16959 - 0.26309) --------------------------------------------- 14 ROUGE-2 Average_R: 0.04832 (95%-conf.int. 0.02575 - 0.07404) 14 ROUGE-2 Average_P: 0.04008 (95%-conf.int. 0.02100 - 0.06148) 14 ROUGE-2 Average_F: 0.04350 (95%-conf.int. 0.02320 - 0.06550) --------------------------------------------- 14 ROUGE-3 Average_R: 0.00626 (95%-conf.int. 0.00129 - 0.01275) 14 ROUGE-3 Average_P: 0.00551 (95%-conf.int. 0.00125 - 0.01106) 14 ROUGE-3 Average_F: 0.00583 (95%-conf.int. 0.00129 - 0.01172) --------------------------------------------- 14 ROUGE-4 Average_R: 0.00000 (95%-conf.int. 0.00000 - 0.00000) 14 ROUGE-4 Average_P: 0.00000 (95%-conf.int. 0.00000 - 0.00000) 14 ROUGE-4 Average_F: 0.00000 (95%-conf.int. 0.00000 - 0.00000) --------------------------------------------- 14 ROUGE-L Average_R: 0.18917 (95%-conf.int. 0.15208 - 0.22554) 14 ROUGE-L Average_P: 0.16072 (95%-conf.int. 0.12931 - 0.19636) 14 ROUGE-L Average_F: 0.17285 (95%-conf.int. 0.13969 - 0.20706) --------------------------------------------- 14 ROUGE-W-1.2 Average_R: 0.11376 (95%-conf.int. 0.09309 - 0.13418) 14 ROUGE-W-1.2 Average_P: 0.15239 (95%-conf.int. 0.12539 - 0.18210) 14 ROUGE-W-1.2 Average_F: 0.12955 (95%-conf.int. 0.10691 - 0.15301) --------------------------------------------- 14 ROUGE-S* Average_R: 0.04052 (95%-conf.int. 0.02377 - 0.05965) 14 ROUGE-S* Average_P: 0.03102 (95%-conf.int. 0.01668 - 0.05095) 14 ROUGE-S* Average_F: 0.03427 (95%-conf.int. 0.01952 - 0.05277) --------------------------------------------- 14 ROUGE-SU* Average_R: 0.07426 (95%-conf.int. 0.05233 - 0.09732) 14 ROUGE-SU* Average_P: 0.05627 (95%-conf.int. 0.03818 - 0.07912) 14 ROUGE-SU* Average_F: 0.06277 (95%-conf.int. 0.04369 - 0.08486) --------------------------------------------- 21 ROUGE-1 Average_R: 0.12268 (95%-conf.int. 0.09798 - 0.14879) 21 ROUGE-1 Average_P: 0.15320 (95%-conf.int. 0.12216 - 0.18730) 21 ROUGE-1 Average_F: 0.13279 (95%-conf.int. 0.10711 - 0.15971) --------------------------------------------- 21 ROUGE-2 Average_R: 0.01529 (95%-conf.int. 0.00592 - 0.02711) 21 ROUGE-2 Average_P: 0.02223 (95%-conf.int. 0.00779 - 0.04143) 21 ROUGE-2 Average_F: 0.01766 (95%-conf.int. 0.00648 - 0.03171) --------------------------------------------- 21 ROUGE-3 Average_R: 0.00146 (95%-conf.int. 0.00000 - 0.00387) 21 ROUGE-3 Average_P: 0.00189 (95%-conf.int. 0.00000 - 0.00500) 21 ROUGE-3 Average_F: 0.00165 (95%-conf.int. 0.00000 - 0.00436) --------------------------------------------- 21 ROUGE-4 Average_R: 0.00000 (95%-conf.int. 0.00000 - 0.00000) 21 ROUGE-4 Average_P: 0.00000 (95%-conf.int. 0.00000 - 0.00000) 21 ROUGE-4 Average_F: 0.00000 (95%-conf.int. 0.00000 - 0.00000) --------------------------------------------- 21 ROUGE-L Average_R: 0.11136 (95%-conf.int. 0.08935 - 0.13337) 21 ROUGE-L Average_P: 0.14091 (95%-conf.int. 0.11120 - 0.17367) 21 ROUGE-L Average_F: 0.12123 (95%-conf.int. 0.09858 - 0.14549) --------------------------------------------- 21 ROUGE-W-1.2 Average_R: 0.07130 (95%-conf.int. 0.05835 - 0.08458) 21 ROUGE-W-1.2 Average_P: 0.14244 (95%-conf.int. 0.11389 - 0.17316) 21 ROUGE-W-1.2 Average_F: 0.09280 (95%-conf.int. 0.07623 - 0.10966) --------------------------------------------- 21 ROUGE-S* Average_R: 0.00977 (95%-conf.int. 0.00502 - 0.01464) 21 ROUGE-S* Average_P: 0.01637 (95%-conf.int. 0.00837 - 0.02539) 21 ROUGE-S* Average_F: 0.01103 (95%-conf.int. 0.00590 - 0.01624) --------------------------------------------- 21 ROUGE-SU* Average_R: 0.03016 (95%-conf.int. 0.02244 - 0.03836) 21 ROUGE-SU* Average_P: 0.05010 (95%-conf.int. 0.03553 - 0.06724) 21 ROUGE-SU* Average_F: 0.03441 (95%-conf.int. 0.02570 - 0.04326) --------------------------------------------- 22 ROUGE-1 Average_R: 0.16619 (95%-conf.int. 0.13350 - 0.20500) 22 ROUGE-1 Average_P: 0.15684 (95%-conf.int. 0.12675 - 0.19079) 22 ROUGE-1 Average_F: 0.15540 (95%-conf.int. 0.12640 - 0.18731) --------------------------------------------- 22 ROUGE-2 Average_R: 0.01970 (95%-conf.int. 0.00940 - 0.03235) 22 ROUGE-2 Average_P: 0.02285 (95%-conf.int. 0.00885 - 0.04183) 22 ROUGE-2 Average_F: 0.01963 (95%-conf.int. 0.00867 - 0.03326) --------------------------------------------- 22 ROUGE-3 Average_R: 0.00267 (95%-conf.int. 0.00000 - 0.00645) 22 ROUGE-3 Average_P: 0.00179 (95%-conf.int. 0.00000 - 0.00439) 22 ROUGE-3 Average_F: 0.00214 (95%-conf.int. 0.00000 - 0.00523) --------------------------------------------- 22 ROUGE-4 Average_R: 0.00000 (95%-conf.int. 0.00000 - 0.00000) 22 ROUGE-4 Average_P: 0.00000 (95%-conf.int. 0.00000 - 0.00000) 22 ROUGE-4 Average_F: 0.00000 (95%-conf.int. 0.00000 - 0.00000) --------------------------------------------- 22 ROUGE-L Average_R: 0.14274 (95%-conf.int. 0.11681 - 0.17154) 22 ROUGE-L Average_P: 0.13564 (95%-conf.int. 0.11091 - 0.16389) 22 ROUGE-L Average_F: 0.13356 (95%-conf.int. 0.11087 - 0.15692) --------------------------------------------- 22 ROUGE-W-1.2 Average_R: 0.08851 (95%-conf.int. 0.07349 - 0.10502) 22 ROUGE-W-1.2 Average_P: 0.13443 (95%-conf.int. 0.10985 - 0.16234) 22 ROUGE-W-1.2 Average_F: 0.10269 (95%-conf.int. 0.08630 - 0.11951) --------------------------------------------- 22 ROUGE-S* Average_R: 0.02048 (95%-conf.int. 0.01204 - 0.03044) 22 ROUGE-S* Average_P: 0.01755 (95%-conf.int. 0.00929 - 0.02714) 22 ROUGE-S* Average_F: 0.01595 (95%-conf.int. 0.00957 - 0.02308) --------------------------------------------- 22 ROUGE-SU* Average_R: 0.04477 (95%-conf.int. 0.03270 - 0.05895) 22 ROUGE-SU* Average_P: 0.04262 (95%-conf.int. 0.02754 - 0.05872) 22 ROUGE-SU* Average_F: 0.03765 (95%-conf.int. 0.02812 - 0.04741) --------------------------------------------- 23 ROUGE-1 Average_R: 0.12235 (95%-conf.int. 0.08927 - 0.15829) 23 ROUGE-1 Average_P: 0.11503 (95%-conf.int. 0.08510 - 0.14914) 23 ROUGE-1 Average_F: 0.11823 (95%-conf.int. 0.08752 - 0.15313) --------------------------------------------- 23 ROUGE-2 Average_R: 0.00681 (95%-conf.int. 0.00000 - 0.01641) 23 ROUGE-2 Average_P: 0.00607 (95%-conf.int. 0.00000 - 0.01473) 23 ROUGE-2 Average_F: 0.00641 (95%-conf.int. 0.00000 - 0.01550) --------------------------------------------- 23 ROUGE-3 Average_R: 0.00000 (95%-conf.int. 0.00000 - 0.00000) 23 ROUGE-3 Average_P: 0.00000 (95%-conf.int. 0.00000 - 0.00000) 23 ROUGE-3 Average_F: 0.00000 (95%-conf.int. 0.00000 - 0.00000) --------------------------------------------- 23 ROUGE-4 Average_R: 0.00000 (95%-conf.int. 0.00000 - 0.00000) 23 ROUGE-4 Average_P: 0.00000 (95%-conf.int. 0.00000 - 0.00000) 23 ROUGE-4 Average_F: 0.00000 (95%-conf.int. 0.00000 - 0.00000) --------------------------------------------- 23 ROUGE-L Average_R: 0.10965 (95%-conf.int. 0.08185 - 0.14105) 23 ROUGE-L Average_P: 0.10383 (95%-conf.int. 0.07819 - 0.13277) 23 ROUGE-L Average_F: 0.10635 (95%-conf.int. 0.07990 - 0.13597) --------------------------------------------- 23 ROUGE-W-1.2 Average_R: 0.06674 (95%-conf.int. 0.05082 - 0.08413) 23 ROUGE-W-1.2 Average_P: 0.10003 (95%-conf.int. 0.07684 - 0.12576) 23 ROUGE-W-1.2 Average_F: 0.07981 (95%-conf.int. 0.06101 - 0.10034) --------------------------------------------- 23 ROUGE-S* Average_R: 0.01001 (95%-conf.int. 0.00430 - 0.01689) 23 ROUGE-S* Average_P: 0.00899 (95%-conf.int. 0.00360 - 0.01568) 23 ROUGE-S* Average_F: 0.00939 (95%-conf.int. 0.00387 - 0.01613) --------------------------------------------- 23 ROUGE-SU* Average_R: 0.02865 (95%-conf.int. 0.01887 - 0.04049) 23 ROUGE-SU* Average_P: 0.02590 (95%-conf.int. 0.01739 - 0.03617) 23 ROUGE-SU* Average_F: 0.02692 (95%-conf.int. 0.01793 - 0.03771) --------------------------------------------- 24 ROUGE-1 Average_R: 0.28540 (95%-conf.int. 0.23134 - 0.34089) 24 ROUGE-1 Average_P: 0.27811 (95%-conf.int. 0.20830 - 0.35585) 24 ROUGE-1 Average_F: 0.26995 (95%-conf.int. 0.21567 - 0.32693) --------------------------------------------- 24 ROUGE-2 Average_R: 0.09309 (95%-conf.int. 0.05629 - 0.13592) 24 ROUGE-2 Average_P: 0.11162 (95%-conf.int. 0.05591 - 0.18063) 24 ROUGE-2 Average_F: 0.09350 (95%-conf.int. 0.05490 - 0.13769) --------------------------------------------- 24 ROUGE-3 Average_R: 0.03075 (95%-conf.int. 0.01081 - 0.05737) 24 ROUGE-3 Average_P: 0.04022 (95%-conf.int. 0.00900 - 0.08575) 24 ROUGE-3 Average_F: 0.03122 (95%-conf.int. 0.00986 - 0.05847) --------------------------------------------- 24 ROUGE-4 Average_R: 0.00860 (95%-conf.int. 0.00000 - 0.02009) 24 ROUGE-4 Average_P: 0.00703 (95%-conf.int. 0.00000 - 0.01639) 24 ROUGE-4 Average_F: 0.00774 (95%-conf.int. 0.00000 - 0.01805) --------------------------------------------- 24 ROUGE-L Average_R: 0.24161 (95%-conf.int. 0.19599 - 0.29178) 24 ROUGE-L Average_P: 0.24108 (95%-conf.int. 0.17423 - 0.31614) 24 ROUGE-L Average_F: 0.23010 (95%-conf.int. 0.18351 - 0.28161) --------------------------------------------- 24 ROUGE-W-1.2 Average_R: 0.14412 (95%-conf.int. 0.11828 - 0.17208) 24 ROUGE-W-1.2 Average_P: 0.22825 (95%-conf.int. 0.16623 - 0.30239) 24 ROUGE-W-1.2 Average_F: 0.16917 (95%-conf.int. 0.13670 - 0.20511) --------------------------------------------- 24 ROUGE-S* Average_R: 0.06608 (95%-conf.int. 0.04152 - 0.09971) 24 ROUGE-S* Average_P: 0.08502 (95%-conf.int. 0.04014 - 0.13988) 24 ROUGE-S* Average_F: 0.05949 (95%-conf.int. 0.03719 - 0.08882) --------------------------------------------- 24 ROUGE-SU* Average_R: 0.10433 (95%-conf.int. 0.07587 - 0.14102) 24 ROUGE-SU* Average_P: 0.12555 (95%-conf.int. 0.06607 - 0.19873) 24 ROUGE-SU* Average_F: 0.09434 (95%-conf.int. 0.06742 - 0.12844) omnisky@omnisky:~/software/ROUGE-1.5.5$
相关文章推荐
- [原创]测试环境搭建虚拟机工具介绍
- 前端开发工具即测试环境搭建
- 分享几款一键搭建php本地测试环境工具
- ADB工具安装对真机进行Monkey测试的环境搭建
- 搜狐云景客户端工具评测之WordPress的搭建
- 分享几款一键搭建php本地测试环境工具
- 发现一个工具,可以将psd 导出成 html,搭建nodejs环境测试
- wifi专业评测扫描工具,测试,信道评级 求试玩,求评侧~~~~~~~~~
- 省时的浏览器同步测试工具browser-sync服务搭建
- FaceBook ATC 弱网测试工具环境搭建
- 如何搭建testlink工具管理测试用例等
- Siege压力测试和评测工具
- T-Plan测试过程管理工具技术摘要
- appium(跨平台自动化测试工具) windows下环境搭建
- 压力测试和评测工具
- Windows Mobile 5.0程序测试工具平台的搭建
- AngularJs专用测试工具Protractor 搭建PhoneCat项目的开发与测试环境
- Android在Eclipse工具搭建测试框架方式
- 谈软件自动化测试工具的评测方法