1 Comment

Automated Testing in an Agile Environment

By Nadine Parmelee, Senior Quality Assurance Engineer

Nadine-Parmelee1-bWhen it comes to software quality and reliability, there are many benefits to be gained from a switch to an agile development environment. Agile helps teams stay focused; it helps them deliver a quality product more quickly. It drives efficiency and leads to improved results throughout the software production process.

Automated Testing: A Must-Have in an Agile Environment

An agile environment requires automated testing. And with more and more development teams moving to agile processes, the need for automated testing has grown exponentially. Agile test cycles tend to limit the amount of testing that can be accomplished, so getting a good regression set of tests automated is more crucial than ever for you to reach your goals of increasing product quality and reducing costs associated with product defects. Continue reading

Leave a comment

Making Automated Tests Data Driven

By Nadine Parmelee

Nadine-Parmelee1-bSo you’ve decided to start an automation project and your team has taken the time to do the tool analysis. Let’s also assume you’ve gotten your team trained on the tool of choice. What comes next? Now is the time to sort out the best way to set up an automation project for the cleanest data and project timeline. Hopefully, the automation and manual test teams have been analyzing which tests should be automated first, and now the automation scripting can begin. Many projects start with automating the manual test cases as they are written and then later expanding on those tests and making them more efficient. I would suggest that there is an even better way to start the automation project.

When analyzing the tests to be automated, analyze the data used for those test cases as well. Are there different expected results or different application features for different data types or options used in the test cases? Does selecting one set of data options create a different selection of data options for another field or dialog? Do the manual tests only cover a limited amount of test data? Now is a good time to enhance these tests and determine how the automation project can enhance your overall testing effort. For situations where the manual tests cover the basic application functionality but a limited set of data points, data driven tests may be a key benefit to your automation effort.

An example of this could be a manual test case for an application that has different data choices presented if a different region is selected:

Region: North America
Size Options: 2, 4, 6, 8, 10, 12, 14, 16, …
Color options then based on size selected

Region: Europe
Size Options: 34, 36, 38, 40, 42, 46, 48, …
Color options then based on size selected

Very often the manual tests will only hit one or two of the size options and some of the color options that flow down the decision tree from there. The automated tests can validate each region, size and color combination in probably the same amount of time as the more limited manual tests. This can be a quick expansion of the testing that is already planned which yields a higher return on investment as well as an increased quality level. This type of automation also frees up the manual testers to concentrate on other investigative testing that can be done. Try data driven tests in your automation project, and let us know how much time and effort you saved.

Leave a comment

Analyzing and Selecting the Right Software Automation Tool

By Nadine Parmelee, Senior Quality Assurance Engineer

Nadine-Parmelee1-bWith all the tool choices available today, selecting an automation tool is a project all on its own. There are a number of variables to consider when selecting an automation tool, as the upfront cost of the tool is not the only consideration. There are potential costs in tool configuration requirements as well as resource training. Next you have to identify the resources you need and how long they will be needed for your project. Taking the time to do the research for selecting the right tool is essential for both tactical and strategic success. Below is a list of elements to take into consideration when researching automation tools:

  • Is this tool recommended for the programming language used for your product?
  • Initial cost per user of the tool – Is it a “per seat” cost or is there a “site” cost available?
  • What are the maintenance costs of the tool?
  • What are the configuration costs to interact with defect and manual test case tools?
  • How many systems the tool can be utilized on?
  • Are there any training costs?
  • Is customer support provided by the tool vendor?
  • What is the expandability for the future for platforms and environments?
  • What expectations should the tool fulfill for test coverage?

We have helped a number of clients with their tool analysis and have seen firsthand how there is no single solution available for everyone. In some cases clients have wanted to use tools they already had purchased corporate licenses for, and we suggested a different tool because of application compatibility or the amount of time that would be involved to create workarounds to best fit the tool for their situation. Some clients look for “free” open source tools thinking their return on investment will be greater, yet they neglect to account for configuration costs and the costs of training involved in getting the resources up to speed. Moreover, there is the need to provide back to the open source community any changes made with the code that might help others; again a time-consuming effort to comply with while still getting the product tested.

To ensure that your team selects the right tool, make sure they are asking the questions that are most relevant and essential to your requirements. This will advance the effort forward without losing non-essential time exploring too many options. I’d recommend that a BRD (business requirements document) for the tool evaluation effort be created up front. This doesn’t necessarily need to be overkill, but collectively focusing stakeholders on requirements is worth the effort in order to keep the business and technical team in sync with expectations. Having a clear understanding of your automation project needs defined and understood is always a best practice. I also recommend that the selection process include obtaining evaluation copies of the potential tools and give your team time to evaluate them. Some tools might be eliminated from consideration quickly while others may necessarily require more time to evaluate.

In my opinion, one of the best tool evaluation methods is to take one of the most complex tests expected for the automation project and try to create those automated scripts with the evaluation packages. Make sure your project team keeps the primary automation project goals in focus; if data creation is the goal of the automation project, an automation tool that doesn’t work with a specific GUI screen or control may not be out of the running. If user interaction is the primary concern, a tool that works great with the GUI but not so great with your database structure may still be a viable candidate. It is important to analyze the strengths and weaknesses of the tools against your project needs and pick the one that will be the most efficient for your situation. Your automation project requirements may get adjusted along with the tool evaluation process as your team evaluates both together.

Test Automation is essentially a strategy that provides acceleration to market with confidence, if well executed. Taking the time to plan well and select the right tool or tools is critical for that confidence. Remember, once you take steps toward test automation, you should consider this effort as part of your ongoing quality assurance ecosystem. You will want to ensure you keep your automation scripts up to date as your product changes over time.

Are you a Developer or a Tester?

by Nadine Parmelee (Definitely a Tester)

In the current business environment, companies are looking for ways to run leaner and meaner, and sometimes this means some people are working multiple roles. The line between Developer and Tester has been thin in the past and nowadays the line is even thinner. It’s not uncommon for an applicant to show up for an interview of what appears to be a QA job and get tested on how much Java they know.  Historically, QA and testing work has been one path in to development and since automation work is a mini development project of its own, it’s easy to see how companies are blurring the lines even more these days.

The fact is, though, there is a difference between developers and testers, especially in how they approach a project and how they think about a project. Assuming a project starts with good requirements, a good developer will look at those requirements and start to think of how they would build the code and the functionality to meet them. A good tester, on the other hand, starts to think about the questions the requirements don’t answer, the “what if the user does this?” scenarios. A developer tends to look at the “happy path” of the code when testing it, and it’s easier for them to miss issues because they build the code based on what they expect to happen next. A tester tends to be more contrarian, watching for the unexpected—and trying to make the unexpected happen.

Generally we tend to be more suited to one role or the other, but there is some role crossover. A good developer needs to be able to test their work to a certain extent, and it helps a good tester to be able read code and step through it to find and debug an issue. It is pretty rare to find a really good developer who is also a really good tester. It happens, but it’s the exception, not the rule. Both roles are critical to a quality software project and it is important to make sure your project team has strengths in both roles.