比较Jmeter、Grinder和JAVA多线程本身压力测试所带来的性能开销

1. 测试环境

jmeter版本 ：jmeter 2.4

grinder的版本 ： Grinder 3

JAVA的版本：JDK 1.6

2. 测试代码

Jmeter测试代码

public class Sampler {
public void test() {
return;
}
}

public class JmeterTest extends AbstractJavaSamplerClient {
Sampler sampler;

@Override
public SampleResult runTest(JavaSamplerContext context) {

SampleResult results = new SampleResult();

results.sampleStart();

sampler.test();

results.sampleEnd();

results.setSuccessful(true);

return results;

}

@Override
public void setupTest(JavaSamplerContext arg0) {
sampler = new Sampler();
}
}

grinder测试代码

public class Sampler {

public void test() {
return;
}

}

# Test.py
#
# A minimal script that tests The Grinder logging facility.
#
# This script shows the recommended style for scripts, with a
# TestRunner class. The script is executed just once by each worker
# process and defines the TestRunner class. The Grinder creates an
# instance of TestRunner for each worker thread, and repeatedly calls
# the instance for each run of that thread.

from net.grinder.script.Grinder import grinder
from net.grinder.script import Test
from sampler import Sampler
test = Test(1, "Sample")
class TestRunner:

# This method is called for every run.
def __call__(self):
mySampler = test.wrap(Sampler())
mySampler.test()

Java本身多线程

public static void test(int numOfThreads, int times) throws InterruptedException, ExecutionException {
ExecutorService executor = Executors.newFixedThreadPool(numOfThreads);
final Sampler sampler = new Sampler();
List<Future<Long>> results = new ArrayList<Future<Long>>();
for (int i = 0; i < times; i++) {
results.add(executor.submit(new Callable<Long>() {
@Override
public Long call() throws Exception {
long begin = System.currentTimeMillis();
sampler.test();
long end = System.currentTimeMillis();
return end - begin;
}
}));
}
executor.shutdown();
while(!executor.awaitTermination(1, TimeUnit.SECONDS));

long sum = 0;
for (Future<Long> result : results) {
sum += result.get();
}

System.out.println("---------------------------------");
System.out.println("number of threads :" + numOfThreads + " times:" + times);
System.out.println("running time: " + sum + "ms");
System.out.println("TPS: " + (double)(100000 * 1000) / (double)(sum));
System.out.println();
}

3. 测试结果

10个线程 100000次运行

--	TPS
Jmeter	50426.10
Grinder	290275.76
Java threads	2.5E7

20个线程 100000次运行

--	TPS
Jmeter	49215.02
Grinder	225402.91
Java threads	2.5E7

50个线程 100000次运行

--	TPS
Jmeter	29312.61
Grinder	212242.13
Java threads	2.5E7

100个线程 100000次运行

--	TPS
Jmeter	29031.03
Grinder	245507.22
Java threads	2.5E7

200个线程 10000次运行(这里减少了一个0)

--	TPS
Jmeter	28039.87
Grinder	232801.77
Java threads	2.5E7

300个线程 10000次运行

--	TPS
Jmeter	27208.16
Grinder	236537.10
Java threads	1818181.81

1000个线程 10000次运行

--	TPS
Jmeter	27208.16
Grinder	236537.10
Java threads	2.5E7

4. 结论

可以看出Jmeter的本身性能开销是很大的，只适合一般应用的性能测试
Grinder在测试的时候发现上下文切换比较严重，而可能是因为内部机制导致的开销较大的，当然如果测试memcache肯定是不适合的，但一般的应用测试基本上没有问题
JAVA多线程本身并发框架性能开销也是有的，但是比较低，适合要求较高的性能测试，如对redis和memcache构建的应用进行压测