Performance Testing and my Learnings

For the last 2 months(actually part time) i am working on performance testing of my product and have learnt quite a bit in this short time. Here are my observations and suggestions for the people who do performance testing .

1. Stress / Load Testing selection : First and foremost thing is know what u exactly want from performance testing . Do u want to know the metrics where your product is going to break or u want to know how faster ur application or site is loading . My objective was mainly to see where my product is going to die down after keeping the load. Keeping a very high load can crash the server , but one should know at what load the server actually crashes . And now comes the second point.

2. Tool selection : Selection of tool is as important as the testing itself .Search for different tools , choose a few which suite your economic and functional requirements. Try each one of them , know the tool which suites you better and then start your load test. Due to some economic or company constraints , my first choice of tool was my a forced one and i did not have much of a choice , which indeed lead to some improper results. The first tool i worked on (dont want to mention the name of the tool) was buggy and was an half working on . It was generating the load on the application , but the functionality wise part was failing . This lead me to go for Jmeter which was acutally used by one of my working products customer . And Jmeter suited me well .So i learnt something from the mistake i did . I could have tried different tools before itself and could have avoided wasting lot of time . Even reading some good reviews on various tools and some good advice would have helped .

3. Network Hardware : For any performance testing , one should have some dedicated machines . Otherwise the results that show will be mostly wrong . Assume u are testing some java application x on some system and you are testing the amount of memory it going to occupy and are also documenting when your application breaks . And you have some java based applications running on that machine . The other java applications that are running on that machine can impact the output of your application . The output results might be wrong . It is always better to have a separate machines for testing , ie different machine for load generator , application and also for database if needed so . Also See that there is sufficient space on the system , the logs are going to occupy lot of size .

4. Dedicated Network : See that the machines which are used for performance testing are connected well . There are n number of factors which can stop or impact your performance testing . System going out network , human failures like some one unknowingly stopping the machine , slow networks , packet looses , etc should not be present

5. Dont report or publish the results just be testing for one time .Test for a few times and check the outputs , take an average of the best results and publish them .

How to Run Jmeter in command line mode

JMeter can be run in 2 different modes

1. GUI based
2. Non-GUI based

While we can use Start in the GUI Mode to start the tests , the test can also be run from command prompt

1. Open a command prompt
2. Go to the folder where Jmeter is present and go in side bin
3. Now type java -jar ApacheJMeter.jar -n -t

JMeter and OutOfMemory

I have been working on Jmeter for a week now. The tool is tough and good . There is lot to learn in this tool . Although i have faced quite a few problems to keep this running , the main one that troubled me was the OUTOFMEMORY error which really frustrated me well.

My requirement was to regenerate an out of memory error which a customer of our product was getting .That was a tricky one as we were not getting it here . So started my load test on my application . My intial requirement was to run 25 threads and see whether i can get outofmemory in my application. I recorded my application, got the setups ready and started the JMeter run .

I got an Out of Memory 1o minutes after the Jmeter got started.

Things i did starts from here

1. Increased the JVM head size settings in jmeter.bat file present under jmeter\bin location. First time i increased the settings from 512m to 1024m . Saved and stated running the Jmeter .Same OutofMemory again after 1 hour of running

Line to change in the bat file "set HEAP=-Xms512m -Xmx512m " .
Xms - starting RAM the jvm will take
Xmx - maximum RAM that is allowed for the HEAP.

2. Now i changed my test machine from 2GB Ram to 4 GB Ram and tried to allocate 2048m to it .But could not succeed as i was not able to start the Jmeter with the following error.
" JVM could not be initialized."

Now so had to reduce the Heap allocation size . Had it set to 1536m and ran it .It ran successfully for 1 hr and again the same problem .

3. Now the final thing i did was changing the 32 bit machine to 64-bit machine . This seems to be running well .

Will update the post if i run into some problem again

Also few more points

Listeners used in Jmeter while load testing can take lot of memory which can lead to a OutOfMemory . View Tree Results which is useful for debugging indeed takes more memory .Aggregate Listener and Spline Listener would be better to use .Graph Listener is also better , but this also consumes more memory.