Archive for November, 2008

Nov 20 2008

JCrawler and http_load: Super Easy Stress Testers

Published by Dave under Deploying, Socialbrowse

We’re doing a bit of load testing on an upcoming feature for Socialbrowse and I thought I’d write about two tools I’ve found to be incredibly helpful and easy to use.

HTTP_LOAD

The first is http_load.  With http_load you can send many requests at a time to load test your server in just a single process from your own machine.  It lets you know what percent of the requests were returned successfully and how fast they were returned on average.   Its of course open source and you can get it here.

Its very easy to install.  Untar the download, cd into the directory and simply run:

make
sudo make install

You first need to create a file (I call it urls) with a return separated list of the urls you want to load test.  Then run http_load with the appropriate flags.  Here are my abbreviated flag notes:

parallel - number of pages to keep open at a time
rate - number of pages to open per second
checksum - verify that the result is correct
fetches - quit after this many fetches
jitter - vary the rate by about 10%
seconds - quit after so many seconds

To see the full directions on using http_load you can view its man page with

cat http_load.1

I use the following command to keep 10 connections open, connecting at a maximum speed of 5 connections per second for 50 seconds.

http_load -parallel 10 -rate 5 -jitter -verbose -seconds 50 urls

Its especially helpful to run http_load from a productions server (on a different internet connection) and while its runing browse your site from your personal machine to get a feel for how slow things will be under your stressed conditions.
 

JCrawler

JCrawler is great in that it doesn’t just hit a list of pages, but crawls your site.  Especially if your site supports caching, single or few page load testing may not be what you want.  In that case JCrawler would be more effective as it would traverse many more pages on your site.  Download it here.

Here are my install notes:

  1. Untar and cd into the folder.
  2. cd dist
  3. Edit conf/crawlerconfig.xml to customize under your own settings
    (this file has inline comments so you should not have a problem
    to figure-out how things work).
  4. chmod +x run.sh
  5. ./run.sh

They have already pointed out several missing links and needed optimizations for us.  Hope it helps you too!

No responses yet