By John Cavazos, Senior Performance Test Engineer
At one time, it was very common for companies to use a homegrown solution to performance test their software. This was mainly due to two factors:
- the lack of available, cost-effective commercial testing tools
- the complexity of their software
The situation is different today as it is common to find excellent, free, open-source tools online. These high-quality, stable, easy-to-use tools don’t have the inherent maintenance costs associated with using homegrown solutions.
There are now several industry standard tools available that can be used to performance test most, if not all of the features a company might need. What isn’t readily available can typically be added to these tools with minimal effort due to their open-source nature. Here are a few I have used:
SoapUI (http://www.soapui.org/)
Soap UI is a great tool from SMARTBEAR. It was primarily developed for performance testing web services that use the SOAP and REST protocol. It has since expanded capabilities to enable testing of many other protocols and technologies.
It can automatically build test suites using the web service’s WSDL and can even make a functional mock web service itself based on that WSDL. It comes in both freeware and paid pro versions, though the free version is typically sufficient for most applications.
SOAP is falling out of favor in the industry, though, and most companies are moving towards RESTful applications. Fortunately, SOAP UI also has features to test REST as well as other protocols and services such as JMS, JDBC and HTTP. It is fairly easy to use and does a great job.
JMeter (http://jmeter.apache.org/)
JMeter is a powerful software performance testing tool from the Apache Software Foundation. It is an open-source tool with plug-in support. This means any missing functionality can easily be added in-house using its framework, by either modifying an existing feature or plug-in or creating a new one.
On the web, there are also many free plugins available to accomplish a variety of tasks and support additional protocols not included out of the box. JMeter is a java application with an easy-to-use UI, but it can also be run in a server mode on several machines from a remote client.
This allows JMeter to run more load from a number of machines while collecting the data in a central location. JMeter can create very complex tests with relative ease and quickness.
It supports out of the box:
- Web – HTTP, HTTPS
- SOAP
- FTP
- Database via JDBC
- LDAP
- Message-oriented middleware (MOM) via JMS
- Mail – SMTP(S), POP3(S) and IMAP(S)
- MongoDB (NoSQL)
- Native commands or shell scripts
- TCP
Apache Bench (http://httpd.apache.org/docs/2.2/programs/ab.html)
Apache Bench is another good tool from the Apache Software Foundation. It’s used for high-load, http-based requests. It comes standard with the Apache HTTP Server. It is not as robust as some of the other tools, but what it lacks in features it makes up for in speed.
It can generate a large amount of load in a very efficient manner. Keep in mind, though, that it is single-threaded, so if you have multiple CPUs/cores, you may want to run multiple instances of the application to generate even more load. One instance is usually enough to generate a substantial amount of load.
Tuning your client and server machines
One thing to keep in mind when performance testing from a Windows, Linux or Unix machine is that there are some settings put into place to limit the amount of load a machine can generate. These are generally used to limit malicious misuse of servers by hackers and spammers, but this can be detrimental to performance testing. To use the machines as performance test clients or servers, you have to make some adjustments to your machines. This is a topic I will cover in a future post.
