One of the things I've been thinking about forever, but never had the time to learn and set up a solution I could stick with, is a load testing framework for various purposes - web sites, databases, custom protocols, etc. Once again, it's time to take a look. Maybe this time it could be final..

Most people I know just try 'ab' (ApacheBench) on a single URL to get a feel of the performance - it's easy, and at least will let you find out if something will bomb fast under load. However, it's also way too simplistic for anything real.

I guess we can also rule out the other extreme, Mercury LoadRunner. I mean, sure, it's powerful, but who has the time and/or money for that?

Microsoft used to have two tools that were pretty decent, WAST and InetLoad 2.0, but I can no longer find them. Anyway, for a Linux and Mac OS X user, they're not very helpful..

The same can be said for OpenSTA, which looks like it's trying to implement a LoadRunner-scale program as an open source project. Tried it once, seemed powerful but difficult, but again, it's Windows only.

I think this leaves me with two alternatives: Apache/Jakarta JMeter, which seems to have progressed a lot over the years, gradually turning from a simple GUI on top of a simplistic load generator to something that may actually be very useful, and The Grinder, version 3 of which I've recently heard only good things about. I haven't tested the latter myself yet, but I do wonder whether it might actually be the better choice.

If I want to have an easy way of setting up simple test scenarios (via a web proxy, for example), run them from a distributed client network to really generate high loads, and on the other hand, also run smaller loads as a part of a scheduled build/deploy/test/report cycle automatically with results going info an HTML format report, which is the better choice?

At least Robert Gash seems to prefer Grinder.

Comment by oa on Wed, 03 Dec 2003 23:51:16:
JMeter has a Non-GUI mode which may be useful for unattended tests.. I haven't yet found how the same is done with The Grinder (I'm just reading docs for now), but on the other hand, being able to just call Java code from the Jython test scripts means I could easily test EJB interfaces without accessing HTTP at all. That can be done with JMeter too, but probably requires more coding (by developing a custom Request plugin)