7 rules for (Agile) Test Automation

29 Jul, 2013

In a previous project I had to compete against the established experts when trying to introduce a sensible test automation approach.
The reasoning why not to work with suitable tools was mainly based on the fear of change, the unwillingness to automate and the build up idea that when you do automate, the only way to overcome problems is by having expensive licenses.
There are several good reasons why you should automate, but most important is that the team is confident about the quality of the product being developed and gets reliable and fast feedback when making changes.

The specialists on a team should be capable of self-sufficiently building a product and dealing with problems, without having to be dependent on external experts. When working in an agile production environment service level agreements aren’t going to help you, when response times are two to three weeks. Whenever a problem occurs you need to be able to respond immediately, otherwise it will block your entire process flow.
Furthermore agile is all about transparency and collaboration. All skills should be bundled; collaboration is the key to a common level of knowledge and a shared understanding.
Most proprietary tools just don’t embrace the idea of collaboration, licenses are to expensive for tools to be made available to all team-members and often require tool specific programming skills.
When working out these thoughts I came up with the following rules for agile test automation:

  • 1. When working in an Agile environment Automation is of vital importance to keep up with the pace of the project, relying on only manual testing is no option.
  • 2. Automation must be possible within the multidisciplinary team, no external expertise should be required to create or maintain tests.
  • 3. A test tool should be flexible, making it easy to respond to change in terms of environmental, data and technical constrains.
  • 4. Creating automated test cases should be relatively simple and intuitive.
  • 5. Tests should be readable and understandable (for anybody). Documenting and formatting tests in unambiguous and ubiquitous language supports this.
  • 6. Tests should be easy accessible and interchangeable between team-members and preferably also for other stakeholders. Anyone in the team should be capable of adding, viewing and running tests.
  • 7. Test execution should be fast and reliable, since fast feedback is your main objective.

In the process of tool selection these rules can guide you in the right direction. Collaborative test automation tools like FitNesse, Cucumber and Twist support these concepts.
Keep in mind that the tool you decide to use is secondary to the process you follow. In the end critical thinking, discipline, commitment and collaboration are the most important drivers behind successful test automation.

Cirilo Wortel is an Agile Test Consultant at Xebia
Newest Most Voted
Inline Feedbacks
View all comments
George Dinwiddie
George Dinwiddie
9 years ago

Cirilo, rather than saying “manual testing is no option” I would suggest “manual testing is insufficient.” The goal is not to eliminate manual testing, as it is to automate routine testing of known expected behavior, both for the fast feedback you mention but also to allow time for manual testing for unanticipated behavior.

Kenneth Truyers
Kenneth Truyers
9 years ago

Very good post!
I think indeed that collaboration and elimination of boundaries and bureaucracy is the way for a fast-moving industry (such as IT). Simplicity and transparency should be at the core of every process.
One point of critique though:
I fail to see what licensing costs and proprietary tools have to do with the whole story. While it’s true that if you don’t make tools available to everyone, it can have an impact on productivity and collaboration, but that’s true for non-agile teams as well (that’s even true in every branch, not just IT).
Apart from that I’d like to hear the reasoning behind the statement that proprietary tools don’t embrace the idea of collaboration. In my opinion, this has nothing to do with proprietary, closed source, open source, … For example, if I look at Team Foundation Server, TeamCity and Jenkins, I can see equal value delivered and equal collaborative features (all with a different accent on the process). These three all have a different licensing model.

Chad Wathington
Chad Wathington
9 years ago

A great book that embodies a bunch of the concepts listed here is “Specification By Example” by Gojko Adzic.
Test automation is hugely valuable (when done well) as part of a larger team approach to quality.

9 years ago

The article is well written. I too agree with you that agile approach is the latest trend in software testing field. In agile testing the testers adapt change easily. You can see more about agile testing in this blog –

Explore related posts