By Nadine Parmelee, Senior Quality Assurance Engineer
With all the tool choices available today, selecting an automation tool is a project all on its own. There are a number of variables to consider when selecting an automation tool, as the upfront cost of the tool is not the only consideration. There are potential costs in tool configuration requirements as well as resource training. Next you have to identify the resources you need and how long they will be needed for your project. Taking the time to do the research for selecting the right tool is essential for both tactical and strategic success. Below is a list of elements to take into consideration when researching automation tools:
- Is this tool recommended for the programming language used for your product?
- Initial cost per user of the tool – Is it a “per seat” cost or is there a “site” cost available?
- What are the maintenance costs of the tool?
- What are the configuration costs to interact with defect and manual test case tools?
- How many systems the tool can be utilized on?
- Are there any training costs?
- Is customer support provided by the tool vendor?
- What is the expandability for the future for platforms and environments?
- What expectations should the tool fulfill for test coverage?
We have helped a number of clients with their tool analysis and have seen firsthand how there is no single solution available for everyone. In some cases clients have wanted to use tools they already had purchased corporate licenses for, and we suggested a different tool because of application compatibility or the amount of time that would be involved to create workarounds to best fit the tool for their situation. Some clients look for “free” open source tools thinking their return on investment will be greater, yet they neglect to account for configuration costs and the costs of training involved in getting the resources up to speed. Moreover, there is the need to provide back to the open source community any changes made with the code that might help others; again a time-consuming effort to comply with while still getting the product tested.
To ensure that your team selects the right tool, make sure they are asking the questions that are most relevant and essential to your requirements. This will advance the effort forward without losing non-essential time exploring too many options. I’d recommend that a BRD (business requirements document) for the tool evaluation effort be created up front. This doesn’t necessarily need to be overkill, but collectively focusing stakeholders on requirements is worth the effort in order to keep the business and technical team in sync with expectations. Having a clear understanding of your automation project needs defined and understood is always a best practice. I also recommend that the selection process include obtaining evaluation copies of the potential tools and give your team time to evaluate them. Some tools might be eliminated from consideration quickly while others may necessarily require more time to evaluate.
In my opinion, one of the best tool evaluation methods is to take one of the most complex tests expected for the automation project and try to create those automated scripts with the evaluation packages. Make sure your project team keeps the primary automation project goals in focus; if data creation is the goal of the automation project, an automation tool that doesn’t work with a specific GUI screen or control may not be out of the running. If user interaction is the primary concern, a tool that works great with the GUI but not so great with your database structure may still be a viable candidate. It is important to analyze the strengths and weaknesses of the tools against your project needs and pick the one that will be the most efficient for your situation. Your automation project requirements may get adjusted along with the tool evaluation process as your team evaluates both together.
Test Automation is essentially a strategy that provides acceleration to market with confidence, if well executed. Taking the time to plan well and select the right tool or tools is critical for that confidence. Remember, once you take steps toward test automation, you should consider this effort as part of your ongoing quality assurance ecosystem. You will want to ensure you keep your automation scripts up to date as your product changes over time.