The future of testing

By Kingston
Duffie

September 1, 2009 —

What if testers were able to
somehow work ahead of developers? I don’t mean agile testing, I mean
anticipating builds and developing solid test plans well ahead of time. What if
instead of developers sending builds to testers, and testers sending bugs to
developers, we had testers sending automated tests to developers, and developers
shipping products when those tests passed on a candidate build? If we had all
this, we would be looking at the future of testing.

You are probably thinking that this is completely
unfeasible. How could we possibly get all of the testing automated in time to
give to the developers while they are still creating the features? Or you might
be thinking that this is crazy, because you can’t trust developers to release
the product themselves! But bear with me. Let’s consider the fundamentals of
this concept before we start to deal with the issues.

In this new process, things get very efficient.
Testers spend their time designing great tests. Since that is all they are
doing, they are going to produce more tests with better coverage. Developers are
going to be a lot more efficient, too. They have tests to tell them immediately
when they break something, rather than dealing with the vagaries of bug reports
that come much later.

In most
cases, test failures are much easier for the developer to analyze given their
understanding of the recent code changes. The most important thing in this
process is the absence of any “looping.” We eliminate the expensive process of
bug fixes requiring new builds that then need to be re-tested, producing new bug
fixes, and so on. The product is ready to release when all of the tests have
been completed and when all of those tests pass.

Also, this new process workflow could solve a common
challenge for many organizations. QA testers often lack sufficient knowledge or
understanding of how the features that they will be testing should work. As a
result, the majority of test development occurs after QA finally gets to the see
the feature in action from an early build. In theory, they should be able to
build a solid test plan—and test cases—from the specification provided by the
developers and use cases from product management. However, this rarely happens:
Either the spec changes or the testers are not knowledgeable enough to
anticipate testing, so they wait.

Given all this, what would it take to make this
process feasible?

1. You must be
able to automate all tests.

2. You
must be able to automate tests without having the product/feature
available.

3. You must be able to
automate in less time than it takes to develop the
product/features.

4. You must be
willing to change.

Let’s take
these one at a time.

Is it
feasible to automate all tests? Perhaps not (although that might be an
interesting debate). Nevertheless, let’s suppose that you can automate 95% of
your testing. Your team can then handle that last 5% using a smaller ancillary
process.

Is it feasible to
automate feature- and system-level tests without having access to the product?
Theoretically, yes. But we know that in practice one must debug automated tests
much like one debugs software. So the absence of a real working version of the
product or feature is problematic. However, certain virtual technologies now
make it possible to emulate the behavior of a system, so you can specify how a
device will respond and therefore debug your test case in advance of having
access to the final product or feature.

Can test automation happen ahead of development? The
answer is definitely yes. Automated testing tools can dramatically accelerate
the process of automation to the point where the first few tests on a new
feature can be in place in a matter of hours. Additional tests extending that
coverage can be developed in parallel as development proceeds, ensuring that the
developer always has a good set of tests to work with.

By moving
automated testing ahead of development, organizations stand to benefit by
eliminating looping between QA and developers. Today, developers test features
with a positive use in mind—that is, they test simply to prove that a feature
works as designed. QA teams, on the other hand, conduct negative use tests to
see how the feature responds when users use it incorrectly.

In the standard workflow, tests flow from development
to QA—first positive use, then negative use—which leads to looping when defects
are found. However, if QA testers provided developers with automated negative
use tests, developers could then run both types of tests and avoid looping. This
would save developers as well as testers considerable time and effort.

To implement this change to the
workflow, testers and developers need a technology that enables them to easily
transfer, understand and run tests. The resulting asset sharing would not only
eliminate looping, as mentioned earlier, but also lead to efficiencies
throughout the workflow. For example, QA testers would be able to pass on
automated tests to developers; developers would leverage these tests to conduct
feature/unit testing; and then developers would pass these semi-automated tests
to the automation specialists. Automation specialists could easily leverage
these semi-automated test cases (they might not have pass/fail criteria) for
regression-validation testing, eliminating the need to decipher written test
plans or build automated tests from scratch.

Will organizations change their processes? I
understand that these kinds of changes don’t happen quickly. But these changes
will come. As with the advent of agile development, making this change requires
business drivers and technology enablers, both of which exist in the form of
market pressures and automated testing tools. The early adopters will be those
who feel the greatest pain and are willing to take some risks to change. Others
will follow.

The future of
testing is really about stepping outside our comfort zones and helping
developers and testers work together in a new and improved way. It’s about being
innovative behind the scenes and rethinking familiar processes in order to
deliver higher-quality products to market faster.

Kingston Duffie is founder and CTO of Fanfare Group,
which provides software quality assurance products and services.

* Source From : VTB (#1 Software Tester Community In Asia) - Article - The future
of testing - http://www.vietnamesetestingboard.org/zbxe/?mid=download&category=39374&document_srl=275288&act=
by
thehardway