by John Brooks, Vice President of Engineering

A question often asked of QA managers is, “Are we done testing yet?”  In reality, what they are likely asking is, “Is it good enough to make our release deadline?”  In an effort to maintain project delivery schedules, there can be tremendous pressure on QA to authorize releases before the software is truly ready for release. This problem is amplified by newer software application architectures and technologies, such as a SaaS application hosted in the cloud where a release can be practically silent and the end users have no idea that an update has occurred.  This scenario can create a less disciplined “what they don’t know won’t hurt them” or “fix it in the next release” kind of approach to software releases, which in turn can harm a company’s reputation (or worse) if critical defects that negatively affect their users are not caught before a release.

So what is a QA organization to do?  How to best deal with this pressure?  Every software project has its constraints – resources, time, and scope – and more often than not it is in the testing phase where shortcuts are taken to try and keep the project within bounds.  But taking shortcuts in quality is high risk—the expense of fixing it later is many times greater, as analysis over the last several years has repeatedly illustrated.

One of the best ways for QA to ensure that adequate testing is completed and an accurate measure of the software quality is provided is to establish gateways between the software development life cycle (SDLC) phases with entrance and exit criteria at the start of a project.  These gateway checkpoints should occur frequently throughout the SDLC, not only at the end immediately before a major release.

How does establishment of the phase gateways help ensure quality software?  Up front recognition and agreement about the test criteria and plans that QA is executing helps project management more objectively measure project status, software quality and readiness.  For example, items that are usually part of the entry criteria include (in order of importance and other phases in the SDLC):

• Requirements traceability matrix
• Release notes with defect logs
• Identified technical and support resources
• Gateway closure report for previous releases (if applicable)

Common exit criteria include:

• Key component:  minimum metrics for successful transition through the phase gateway, such as maximum number of Severity 1 and 2 defects and overall quality score
• Test scenarios and scripts executed, along with (or as a part of) a test-to-requirements matrix (includes a minimum % pass value)
• List of assumptions and tests that are not applicable
• Test closure report (includes entry release notes)

In summary, defining the SDLC phase transition criteria for a particular project, if done properly, will enhance both the flow and the quality of a project—not burden it down with excessive “hoops to jump through.”  Phase gateways then become an important tool for the team to recognize where they are in the process and how well that process is working for the project.