By Paul Cooper, Senior Performance Test Engineer

Test automation can be a costly time consuming endeavor. Often automation code and test data become entangled, leading to both fragile tests and fragile automation code. Once that happens, new tests will require programming to implement them. This helps us understand why testers now need coding skills. Design changes to the application, such as changing where or how a value is input, will also require updating the automation code and in turn may disrupt the test data.

Data Driven Testing (DDT) can help separate test data from automation code. One advantage of properly designed and implemented DDT is that a Subject Matter Expert (SME) can express the test at a high level just through the input data and the expected output data. This leads to additional benefits:

1. The tests are independent of the implementation.

2. Modifying or adding new test may not require new code.

3. Tests can be written by non-programmers.

4. Tests can be designed before or in parallel with automation implementation.

5. Through reusability, less actual code is required and is therefore code is easier to maintain.

In principal the DDT can be thought of as a table of values where each row is a test containing at least parameterized inputs and expected outputs. Non test data on a row could be a test number, requirements tracker, simply a comment … etc. A single column represents the values for a single input/output test parameter. A natural implementation of the data is a spreadsheet.

In practice the data can be stored in:

• Simple comma/tab delimited text files

• Database tables

• XML/Json

• Whatever format suits your needs

The choice of how the data is formatted and where it is stored comes down to convenience for the test writer and the automation developer. If delimited text files are the input to the automation then a spreadsheet application can be used to write the tests and then exported with the chosen delimiter. An additional benefit of using a spreadsheet includes macros that can be used to calculate expected outputs from the given inputs and validate inputs that are constrained to certain values, thereby reducing the bad data risks.

Database tables may require the test writer to be familiar with DB tools and concepts, unless given a data entry form. An advantage for databases is data from different tables can be combined to make new tests via SQL. A disadvantage for databases would be source control. While XML or JSON file don’t conjure up images of tables, they do provide a way to easily handle complex or long data that can be problematic for spreadsheets.

The automation developer’s task is to present the data to the application in a manner that is consistent with actual use. That may be through a GUI, API calls, a data pipe or something else. The data may simply require operating a single page of a web application or may simulate a more involved end to end test between systems. Since the data format is known, writing a specific driver comes down to performing setup, running through the data, logging results, tearing down. Normally, to test one application there will be many differing sets of tests. There will be many opportunities for code reuse in the test drivers, both from the standpoint of accessing the data to using it within the application.

Using DDT does not guarantee data/code separation. If the test writer and the automation developer are the same person, then application specifics may end up as data. This happen often when dealing with GUI interfaces and especially when automating web applications. Usually it happens in the form of control selectors (CSS) for GUI elements appearing in the data. Even so, DDT is a useful way to decompose tests from automation code.