Leave a comment

Analyzing and Selecting the Right Software Automation Tool

By Nadine Parmelee, Senior Quality Assurance Engineer

Nadine-Parmelee1-bWith all the tool choices available today, selecting an automation tool is a project all on its own. There are a number of variables to consider when selecting an automation tool, as the upfront cost of the tool is not the only consideration. There are potential costs in tool configuration requirements as well as resource training. Next you have to identify the resources you need and how long they will be needed for your project. Taking the time to do the research for selecting the right tool is essential for both tactical and strategic success. Below is a list of elements to take into consideration when researching automation tools:

  • Is this tool recommended for the programming language used for your product?
  • Initial cost per user of the tool – Is it a “per seat” cost or is there a “site” cost available?
  • What are the maintenance costs of the tool?
  • What are the configuration costs to interact with defect and manual test case tools?
  • How many systems the tool can be utilized on?
  • Are there any training costs?
  • Is customer support provided by the tool vendor?
  • What is the expandability for the future for platforms and environments?
  • What expectations should the tool fulfill for test coverage?

We have helped a number of clients with their tool analysis and have seen firsthand how there is no single solution available for everyone. In some cases clients have wanted to use tools they already had purchased corporate licenses for, and we suggested a different tool because of application compatibility or the amount of time that would be involved to create workarounds to best fit the tool for their situation. Some clients look for “free” open source tools thinking their return on investment will be greater, yet they neglect to account for configuration costs and the costs of training involved in getting the resources up to speed. Moreover, there is the need to provide back to the open source community any changes made with the code that might help others; again a time-consuming effort to comply with while still getting the product tested.

To ensure that your team selects the right tool, make sure they are asking the questions that are most relevant and essential to your requirements. This will advance the effort forward without losing non-essential time exploring too many options. I’d recommend that a BRD (business requirements document) for the tool evaluation effort be created up front. This doesn’t necessarily need to be overkill, but collectively focusing stakeholders on requirements is worth the effort in order to keep the business and technical team in sync with expectations. Having a clear understanding of your automation project needs defined and understood is always a best practice. I also recommend that the selection process include obtaining evaluation copies of the potential tools and give your team time to evaluate them. Some tools might be eliminated from consideration quickly while others may necessarily require more time to evaluate.

In my opinion, one of the best tool evaluation methods is to take one of the most complex tests expected for the automation project and try to create those automated scripts with the evaluation packages. Make sure your project team keeps the primary automation project goals in focus; if data creation is the goal of the automation project, an automation tool that doesn’t work with a specific GUI screen or control may not be out of the running. If user interaction is the primary concern, a tool that works great with the GUI but not so great with your database structure may still be a viable candidate. It is important to analyze the strengths and weaknesses of the tools against your project needs and pick the one that will be the most efficient for your situation. Your automation project requirements may get adjusted along with the tool evaluation process as your team evaluates both together.

Test Automation is essentially a strategy that provides acceleration to market with confidence, if well executed. Taking the time to plan well and select the right tool or tools is critical for that confidence. Remember, once you take steps toward test automation, you should consider this effort as part of your ongoing quality assurance ecosystem. You will want to ensure you keep your automation scripts up to date as your product changes over time.


Leave a comment

Understanding Build Server and Configuration Management – Part 1

A two-part series by Morgan McCollough, Senior Software Engineer

Morgan McCollough Bridge360The terms configuration management, continuous integration, and build server are used like buzzwords in the software world today. But how many people really understand what they mean in terms of time, efficiency and money? Most computer science programs in universities don’t talk about the practical application and benefit of these concepts, possibly because they are beyond the purview of the academic science.

In general, software configuration management is the systematic management of software changes both in terms of code and configuration for the purpose of maintaining integrity and traceability throughout the life cycle. In practical terms this can mean a wide variety of things, all the way from a simple source control branching and release management process to CMM level 5.

Continuous integration is the practice of merging all developer working copies of a source tree several times a day into a central development branch for the purpose of building, testing, and providing instant feedback. Implementing some version of one or both of these environments requires the creation of a centralized build server, the system where all code is merged, builds are produced, unit tests are executed, and eventually automated deployments are initiated.

In my line of work, I am in the position to work with a number of different clients and see diverse development environments and teams. In teams larger than two people, the difference between those that use continuous integration and solid configuration management and those that don’t is significant. I think it is very likely that many teams don’t implement continuous integration or a build server because they don’t fully appreciate what can really go wrong or the potential negative effects when they don’t have these systems in place. The time wasted doing builds, fixing deployment configurations, and correcting broken builds when configuration management is not in place is a true hidden cost in software development.

Teams that have these systems in place:

  • Have a more stable code base
  • Tend to be more flexible
  • Can deploy and test in disparate and new environments with minimal effort, and
  • Can reproduce any specific build or release from any point in the history of the application

Many software development teams are familiar enough with the concepts of build servers, configuration management and continuous integration to know they are a good idea, but often decide it’s more trouble than it is worth to implement them. In my next post, I’ll outline the basic steps you can follow to implement continuous integration and configuration management in your development environment.


1 Comment

Why You Should Think of People as Raw Materials

by Phillip Smith

Those of us in the service industry do not always have something tangible to show as a result of our work. Compare us to people who build things (like roads, or boats, or furniture) or to people who refine, grow, weave, or clean (to provide fuels, clothing, or foods) and we’re quickly at a loss to point to anything physical that is the result of our efforts. Extend the comparison a bit further to folks who perform, write, invent, or design, and we still fall short in terms of having a tangible product. However, that does not mean that we, specifically those of us who sell or provide a service, are not building something.

Consider the absolute best outcome of a good day’s work in the service industry: the result is a new or improved “relationship.” Yes, people who work in sales and service are building; they are building relationships.

To the casual observer, it may appear that they are explaining the features of a product, or in the case of something closer to my daily work, deploying high quality software into your business environment. And that is the day-to-day picture of what we do. However, the observer who is looking a bit deeper sees us doing more than filling a role in order to put food on the table. We are building expectations, trust, and ethical standards that result in long lasting high value relationships. In these cases “tangible” falls short of being a good measurement of value because the end result is anything but an end. Solid relationships, new processes, better standards—these have a ripple effect in the way people do business, and even if the work didn’t result in the next greatest invention, it’s still the kind of stuff that changes the world.

I’ve not presented anything revolutionary in this article. I’m hoping however, that I caught your eye with the title and that you’ll be willing to think about your hiring, training, and culture in a slightly different way. You would not assemble furniture using an unreliable design or with poor quality components. You should think of the people in your service company the same way that you think of the materials that you might use to make a salable product, with the exception that you are using these people to build relationships that make your business successful.

Are they the best people you can find? Have you trained them well and set them up for success? Are they true critical thinkers, not satisfied until they’ve found the best way of doing something? Have you set the right example with the relationships that you maintain with your clients and with your employees?

Have you communicated the right message to your employees, so they understand how to win repeat business? A smile and a willingness to view the service from the client’s point of view is a key ingredient to repeat business, and it’s not always something you can teach. The people that do your smiling are doing what they love, caring about the results and invested in the outcome—and they are the equivalent of the grapes that go into a fine wine.


Leave a comment

Minimizing Risk in an Agile Approach

by Phillip Smith

The discussion about which project management approach to use is irrelevant if the chosen approach is not implemented correctly. The IT industry, while relatively young, does have reasonably well thought out software development processes (SDP’s) that are mature and reliable. Taking shortcuts within an SDP is never a good idea.

Whether using agile or waterfall, always make sure your plan includes stable requirements, optimized design, and rigorous testing. You should never tailor out critical deliverables, breeze conditionally through tollgates, or attempt to support truly impossible timelines.  It is better to execute in a reliable and true fashion and finish the project the right way, with quality. Stated another way, project managers should never use a difficult timeline or a low budget to justify chaos.

For example: in an agile approach, if you start with a very large scope and parse it into smaller components, and then execute all of the SDP phases for a single component prior to going in to the requirements for any other part of the scope, you will be running a great risk.  The lack of definition of follow-on components, combined with the finality of having the first component implemented, will certainly lead to integration issues and rework.

Continue reading


Leave a comment

Options for Modernizing Legacy Systems

by Chris Durand

Say you have an existing iSeries (AS/400) or mainframe “green screen” application.  It’s been around for a long time and has a solid, reliable code base based on years (decades?) of testing and real-world use.  It generally works pretty well, and doesn’t change much.

That’s all great until you find that as the business is moving forward it is becoming increasingly difficult to update the application to support the future business direction.  Sure, you’ve found a few creative solutions to work around application limitations so far, but fundamentally you need something other than a short-term fix.

You basically have the following modernization options:

  • Rewrite it using the language/technology/platform of your choice.
    • Pros: Complete flexibility to keep what is working and rework what is not.
    • Cons: Expensive.  Risky (does anyone really know all the business rules buried in decades of code?).  Slow to implement.
  • Convert the code to a new platform using an automated tool.  I’ve not had experience with this, however, I suspect it is cheaper than rewriting, but still very risky.  Plus you probably don’t get the full benefits of a true rewrite. Continue reading