I recently read an excellent article authored by Phillip Howard of Bloor Research.  The article, entitled “Testing is Broken”, describes at a high level some of the institutional and practical challenges to effectively addressing the reasons why testing and software quality assurance remain largely a manual process.  Mr. Howard briefly touches on most of the substantive barriers to widespread test automation tool adoption.

I spend much of my time with executives that are responsible for software quality assurance.  In the case of complex enterprise applications, this translates into the need for end to end, cross platform business process validation.  Sounds complicated?  In actual practice it is even much more complex than it appears.  Why is this?

Enterprise applications (think SAP, Oracle EBS, and other monolithic applications) typically share a number of unique characteristics that make the business process validation process daunting.  Most of these applications are:

• Mission critical
• Highly customized
• Integrated with other applications/technologies
• Change-centric

Often, the tools and processes used to ensure high levels of quality for these high-risk applications aren’t sufficient to accomplish the objective.

In many cases, the approach to resolve this has been to outsource manual and semi-automated (scripted) testing to off-shore outsourcers who help reduce the labor expense.  This is frequently referred to by the end user as “their automation solution”.  Unfortunately there is a notable lack of genuine automation in that approach.

Low cost manual labor isn’t automation.

Senior IT executives will tell me, with a straight face, that their automation solution includes hundreds, and sometimes thousands, of outsourced manual testers.  This means that what we call automation is actually manual, repetitive, and error prone, which translates into severe limits in terms of scope and complexity.

Even the firms that outsource the development of semi-automated “scripted” tests are in danger since the people with the technical skills necessary to develop these scripts rarely understand the applications under test and they are too far removed from the application owners, who aren’t scripters.  The gap between these two skills creates obvious problems.

Furthermore, the people who are developing the manual tests rarely understand the applications under test since they are too far removed from the application owners who don’t know how to write tests or don’t have the time.  Yet these are the people who own the responsibility of ensuring that the applications function as required.

As Philip mentions in his article, the service providers have no incentive to change either dynamic.  They are, however, greatly incented to perpetuate them because they stand to make more money this way.  Because the knowledge of the systems and the test artifacts are vendor-specific, it becomes extremely difficult to manage the provider and ensure that the process is efficient, not to mention have them take ownership of the process and result.  The resultant cost, inefficiency, and lack of tangible results becomes the norm and a vicious cycle of hiring/firing offshore service providers ensues.

Today’s innovative test automation tools tackle the inherent problems with manual testing.  Modern architecture eliminates the need for highly specialized technical skills to create automation manually, putting easy to use technology into the hands of the business users that understand the application.  Testing resources can greatly expand the depth and breadth of the application coverage, resulting in more agile, rapid turnaround and improved overall quality.

The cost of poor quality can have a disastrous effect on a business.  Enterprise applications, often used for everything from finance to HR, manufacturing and supply chain management, measure production outage cost in the millions of dollars per hour.  Customer satisfaction and retention suffer.  Using today’s test automation technology can replace the cost, inefficiency and limitations associated with manual testing and deliver substantial operational and financial benefits.

So don’t be fooled…lowering the cost of manual testing and scripting isn’t automation.