Hadoop 性能测试工具
2014-10-28 09:19
197 查看
su - hdfs
PI Estimator Testing:
hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 10 100
TeraGen/TeraSort/TeraValidate Testing:
1. hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar teragen 10485760000 /user/hduser/input # 10,485,760,000=1000GB=1TB
2. hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar terasort /user/hduser/input /user/hduser/output
3. hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar teravalidate /user/hduser/output /user/hduser/validate
TestDFSIO Testing:
Usage:
hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar TestDFSIO -read | -write | -clean [-nrFiles N] [-fileSize MB]
1. hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar TestDFSIO -write -nrFiles 10 -fileSize 1000
2. hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar TestDFSIO -read -nrFiles 10 -fileSize 1000
3. hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar TestDFSIO -clean
NameNode benchmark (nnbench):
The benchmark can simulate requests for creating, reading, renaming and deleting files on HDFS.
hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar nnbench -operation create_write -maps 12 -reduces 6 -startTime 10 -blockSize 1 -bytesToWrite 0 -numberOfFiles 1000 -replicationFactorPerFile 3 -readFileAfterOpen true -baseDir /benchmarks/NNBench-`hostname -s`
MapReduce benchmark (mrbench):
hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar mrbench -numRuns 50
TestDFSIO, nnbench, mrbench
hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar
TeraGen/TeraSort/TeraValidate , PI Estimator
hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar
Running SparkPi in YARN Cluster/Client Mode
( you should first install spark service with cloudera manager )
spark-submit --class org.apache.spark.examples.SparkPi --deploy-mode cluster --master yarn /opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark/examples/lib/spark-examples-1.1.0-cdh5.2.0-hadoop2.5.0-cdh5.2.0.jar 10
spark-submit --class org.apache.spark.examples.SparkPi --deploy-mode
client --master yarn
/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark/examples/lib/spark-examples-1.1.0-cdh5.2.0-hadoop2.5.0-cdh5.2.0.jar
10
本文出自 “Ilovecat(个人笔记)” 博客,请务必保留此出处http://hj192837.blog.51cto.com/655995/1568673
PI Estimator Testing:
hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 10 100
TeraGen/TeraSort/TeraValidate Testing:
1. hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar teragen 10485760000 /user/hduser/input # 10,485,760,000=1000GB=1TB
2. hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar terasort /user/hduser/input /user/hduser/output
3. hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar teravalidate /user/hduser/output /user/hduser/validate
TestDFSIO Testing:
Usage:
hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar TestDFSIO -read | -write | -clean [-nrFiles N] [-fileSize MB]
1. hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar TestDFSIO -write -nrFiles 10 -fileSize 1000
2. hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar TestDFSIO -read -nrFiles 10 -fileSize 1000
3. hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar TestDFSIO -clean
NameNode benchmark (nnbench):
The benchmark can simulate requests for creating, reading, renaming and deleting files on HDFS.
hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar nnbench -operation create_write -maps 12 -reduces 6 -startTime 10 -blockSize 1 -bytesToWrite 0 -numberOfFiles 1000 -replicationFactorPerFile 3 -readFileAfterOpen true -baseDir /benchmarks/NNBench-`hostname -s`
MapReduce benchmark (mrbench):
hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar mrbench -numRuns 50
TestDFSIO, nnbench, mrbench
hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar
TeraGen/TeraSort/TeraValidate , PI Estimator
hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar
Running SparkPi in YARN Cluster/Client Mode
( you should first install spark service with cloudera manager )
spark-submit --class org.apache.spark.examples.SparkPi --deploy-mode cluster --master yarn /opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark/examples/lib/spark-examples-1.1.0-cdh5.2.0-hadoop2.5.0-cdh5.2.0.jar 10
spark-submit --class org.apache.spark.examples.SparkPi --deploy-mode
client --master yarn
/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark/examples/lib/spark-examples-1.1.0-cdh5.2.0-hadoop2.5.0-cdh5.2.0.jar
10
本文出自 “Ilovecat(个人笔记)” 博客,请务必保留此出处http://hj192837.blog.51cto.com/655995/1568673
相关文章推荐
- Apache自带的性能测试工具ab的使用
- [原创]性能测试工具介绍
- Mercury性能测试工具LoadRunner 8.0试用版下载
- 测试网站各项性能的31 个免费在线工具
- 测试网站各项性能的31 个免费在线工具
- 点评性能测试工具OpenSTA
- 服务器性能测试典型工具介绍
- 认识 p-unit : 一款开源的性能测试工具
- SOAP协议性能测试工具—SOAP Performer 0.1正式发布
- 用31个免费在线工具来测试你网站各项性能
- [原创]webload性能测试工具介绍
- LoadRunner 同开源性能测试工具 OpenSTA 的比较
- LoadRunner 同开源性能测试工具 OpenSTA 的比较
- 认识 p-unit : 一款开源的性能测试工具
- 最全,最强的软件测试资料汇总 (性能测试,性能调优,功能测试,自动化测试,测试管理,测试工具,测试用例设计,缺陷分析预防,前沿测试技术...)
- [java]性能测试分析工具EclipseProfiler
- 开源性能测试工具 - Apache ab 介绍
- 服务器性能测试典型工具介绍
- 测试网站各项性能的31 个免费在线工具
- 服务器性能测试典型工具介绍