Better Automated GUI Testing via Virtual Frame Buffers

Leave a comment

troy-rudolphby Troy Rudolph, Senior Software Engineer

We’ve been assisting one of our clients in the development of a large web application facing many large customers and working with several types of devices for data gathering. As a result, we’ve built a lot of automated tests at all levels: acceptance, functional, integration and unit.

Originally, our acceptance testing environment consisted of:

  • Codeception, primarily the WebDriver module,
  • Selenium, as a browser manager, e.g. to start/stop browsers as needed, and
  • PhantomJS, a headless browser.

We selected PhantomJS because it is a headless browser, e.g. it requires no physical display to run the application and thus has a reputation for speed. For whatever reason, we were unable to make PhantomJS work effectively for us and set out to find another solution.

Unfortunately, there aren’t any other headless browsers of note. So, we started looking for ways to run an existing browser as if it were headless. We settled on Firefox largely because it is more compatible with our testing environment.

To run in a headless environment, Firefox still needs a place to render. In a Unix/Linux environment, the X-Windows Virtual Frame Buffer (Xvfb) provides a memory-based X-terminal and thus no physical display is needed.

Thus, our new acceptance testing environment consists of:

  • Codeception
  • Selenium
  • Xvfb
  • Firefox

Although it took a lot of experimentation, it turns out that this is an easy thing to set up. First, we installed Xvfb on Linux environment. On our Centos system, this can be done as follows.

yum install Xvfb

Next, we created a new service for Xfvb, by adding a new service script to /etc/init.d/ folder. In the Xfvb service script, starting the service is easy. In the start() function, this command below will execute Xvfb as a detached process and create window :4444.

/usr/bin/Xvfb :4444 -ac &>/dev/null &

Next, we installed Firefox. We downloaded the package from https://download-installer.cdn.mozilla.net/pub/firefox/releases/37.0/linux-x86_64/en-US/ and installed it manually using good old TAR.

Finally, the script for our Selenium service, also in /etc/init.d, was modified to establish window :4444 as the default window, and to make Firefox available in the path. Now when Selenium launches Firefox, it will render in window :4444 which exists nowhere but virtual memory.

export DISPLAY=:4444
export PATH=/testautomation/firefox
java -jar /ta/selenium-server-standalone-2.45.0.jar -port 4444 &>/dev/null &

Having established the services, we updated the Codeception configuration file for our acceptance suite (acceptance.suite.yml) to request an instance of Firefox for each test executed.

modules:
  enabled:
    - WebDriver:
      url: 'http://your.app.com/'
      browser: firefox

The results from this effort were very good. The switch to Firefox caused a ten to fifteen percent increase in the elapsed time for our tests. This “cost”, however, is well worth it. The reliability of our acceptance suites is greatly improved. This gives us much better feedback on the health of the application.

Note: Some experimentation was required with releases of Selenium Standalone Server and Firefox to arrive at a consistently reliable combination. For us, Selenium Standalone Server 2.45.0 and FireFox 37 worked the best.

Author: bridge360blog

Software Changes Everything.... Bridge360 improves and develops custom application software. We specialize in solving complex problems at every phase of the software development lifecycle, removing roadblocks to help our clients’ software and applications reach their full potential in any market. The Bridge360 customer base includes software companies and world technology leaders, leading system integrators, federal and state government agencies, and small to enterprise businesses across the globe. Clients spanning industries from legal to healthcare, automotive to energy, and high tech to high fashion count on us to clear a path for success. Bridge360 was founded in 2001 (as Austin Test) and is headquartered in Austin, Texas with offices in Beijing, China.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s