Hands-on Test Automation Tools session wrap up – Part1

Last week we had our first Hands-on Test Automation sessions.
Developers and Testers were challenged to show and tell their experiences in Test Automation.
That resulted in lots of in depth discussions and hands-on Test Automation Tool shoot-outs.

In this blogpost we’ll share the outcome of the different sessions, like the famous Cucumber vs. FitNesse debat.

Stay tuned for upcoming updates!

Test Automation Frameworks

The following Test Automation Frameworks were demoed and discussed

1. FitNesse

FitNesse is a test management and execution tool.
You’ll have to write/use fixture code if you want to use Selenium / WebDriver, webservices and databases in your tests.
Pros and Cons
You can have a good drill down in test results.
You can make use of scenario’s and scenario libraries to make test automation steps reusable.
But refactoring is hard when scenario’s are used extensively since there is no IDE support (yet)

2. Cucumber

Cucumber is a specification tool which describes how software should behave.
You’ll have to write/use step definitions if you want to use Selenium / WebDriver, webservices and databases in your tests.
Pros and Cons
Cucumber forces you to write specifications / tests with scenarios (Behaviour in human readable language).
You can drill down into test results, but you’ll need reporting libraries like Cucumber Reporting
We recommend using IntelliJ IDEA with the Cucumber plugin since it supports Cucumber seamlessly.
Refactoring becomes less problematic since you’re using a IDE.

3. Selenium / WebDriver IDE

Selenium / WebDriver automates human interactions with web browser.
With the Selenium IDE you can record and play your tests in Firefox
Pros and Cons
It can get you started very quickly. You can record and play your tests scripts without writing any code.
Reusability of test automation code is not possible. You’ll need to export it into an IDE to introduce reusability.

Must haves in Test Automation Tools

During the parallel sessions we’ve collected the following must haves in test automation tools.

Testers and Developers becoming best friends

When developers do not feel comfortable with the test automation tool, testers will try to fill the gap all by themselves. Most of the time these efforts result in hard to maintain test automation code. At some point in time test automation will become a bottleneck in Continuous Delivery. When picking a test automation tool consider each other’s needs and pick a tool together. Feeling comfortable in writing and maintaining test automation code is really important to make test automation beneficial.

Separating What from How

Tools like FitNesse and Cucumber were designed to separate the What from the How in test automation. When combining both in those tools, you’ll end up getting lost in details and you’ll lose focus of what you are testing.
Use tools like FitNesse and Cucumber to describe What you are testing and put all details about How you are testing in test automation code (like fixture code and step definitions)

Other interesting tools

• Thucydides: Reporting tests and results (including functional coverage)
• Vagrant: Provisioning System Under Test instances
• Liquibase: Treating database schema changes as ‘code’

Stay tuned for upcoming updates!

 Qxperts. We empower companies to deliver reliable & high-quality software. Any questions? We are here to help! www.qxperts.io