Data Driven Testing – Separating Automation Code from Test Data

Leave a comment

By Paul Cooper, Senior Performance Test Engineer

Test automation can be a costly time consuming endeavor. Often automation code and test data become entangled, leading to both fragile tests and fragile automation code. Once that happens, new tests will require programming to implement them. This helps us understand why testers now need coding skills. Design changes to the application, such as changing where or how a value is input, will also require updating the automation code and in turn may disrupt the test data.

Data Driven Testing (DDT) can help separate test data from automation code. One advantage of properly designed and implemented DDT is that a Subject Matter Expert (SME) can express the test at a high level just through the input data and the expected output data. This leads to additional benefits:

  1. The tests are independent of the implementation.

  2. Modifying or adding new test may not require new code.

  3. Tests can be written by non-programmers.

  4. Tests can be designed before or in parallel with automation implementation.

  5. Through reusability, less actual code is required and is therefore code is easier to maintain.

In principal the DDT can be thought of as a table of values where each row is a test containing at least parameterized inputs and expected outputs. Non test data on a row could be a test number, requirements tracker, simply a comment … etc. A single column represents the values for a single input/output test parameter. A natural implementation of the data is a spreadsheet.

In practice the data can be stored in:

  • Simple comma/tab delimited text files

  • Database tables

  • XML/Json

  • Whatever format suits your needs

The choice of how the data is formatted and where it is stored comes down to convenience for the test writer and the automation developer. If delimited text files are the input to the automation then a spreadsheet application can be used to write the tests and then exported with the chosen delimiter. An additional benefit of using a spreadsheet includes macros that can be used to calculate expected outputs from the given inputs and validate inputs that are constrained to certain values, thereby reducing the bad data risks.

Database tables may require the test writer to be familiar with DB tools and concepts, unless given a data entry form. An advantage for databases is data from different tables can be combined to make new tests via SQL. A disadvantage for databases would be source control. While XML or JSON file don’t conjure up images of tables, they do provide a way to easily handle complex or long data that can be problematic for spreadsheets.

The automation developer’s task is to present the data to the application in a manner that is consistent with actual use. That may be through a GUI, API calls, a data pipe or something else. The data may simply require operating a single page of a web application or may simulate a more involved end to end test between systems. Since the data format is known, writing a specific driver comes down to performing setup, running through the data, logging results, tearing down. Normally, to test one application there will be many differing sets of tests. There will be many opportunities for code reuse in the test drivers, both from the standpoint of accessing the data to using it within the application.

Using DDT does not guarantee data/code separation. If the test writer and the automation developer are the same person, then application specifics may end up as data. This happen often when dealing with GUI interfaces and especially when automating web applications. Usually it happens in the form of control selectors (CSS) for GUI elements appearing in the data. Even so, DDT is a useful way to decompose tests from automation code.

Author: bridge360blog

Software Changes Everything.... Bridge360 improves and develops custom application software. We specialize in solving complex problems at every phase of the software development lifecycle, removing roadblocks to help our clients’ software and applications reach their full potential in any market. The Bridge360 customer base includes software companies and world technology leaders, leading system integrators, federal and state government agencies, and small to enterprise businesses across the globe. Clients spanning industries from legal to healthcare, automotive to energy, and high tech to high fashion count on us to clear a path for success. Bridge360 was founded in 2001 (as Austin Test) and is headquartered in Austin, Texas with offices in Beijing, China.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s