Robot Framework – The unsung hero of test automation

The open source Robot Framework (RF) is a generic, keyword- and data-driven test automation framework for acceptance test driven development (ATDD). As such it stands alongside similar, but more well-known frameworks, like FitNesse, Cucumber, et alia. The (relative) unfamiliarity of the testing community with the RF is undeserved, since the RF facilitates powerful and yet simple test automation against a variety of interfaces and features some distinct advantages when compared to those other frameworks.

In a series of blogposts, we would like to make a case for the Robot Framework, by showing its greatness through a number of hands-on examples from my upcoming workshop. Next to demonstrating its advantages and strengths we will also expose some of its drawbacks and limitations, as well as touch upon certain risks that flow from harnessing some of its unique features.

Our first three posts will give an introductory overview of the RF, laying the conceptual foundation for the remainder of the series. Therefore these three articles will not concentrate on practical, hands-on examples or instructions, but instead have a more theoretical feel. Moreover, several of the fundamental concepts laid out in them, are applicable not only to the RF, but to most (if not all) test automation frameworks. Consequently, these first three posts target those that miss a basic understanding of test automation (frameworks) in general and/or of the RF in particular. The remainder of the series will be also of interest to more seasoned automation engineers.

We will first look into the basic architecture that underlies the framework and discuss the various components that it is being comprised of. In the second post we will discuss the nature of the keyword-driven approach that the RF entails. The third post will detail a typical test automation work flow.

For a first-hand experience of the pros and cons of the RF, you might want to join the first Robot Framework meetup in the Netherlands.

Robot Framework stack

The RF is written in Python. Consequently, it runs on Python, Jython or IronPython. The framework can thus be used with any OS that is able to run any of these interpreters, e.g. Windows, Linux or OS X. Unless you have a specific reason to do otherwise, the RF should run on Python. A typical situation that would require e.g. Jython, would be automating against a Java application or implementing your own RF test library in Java (more on this in a later post). A disadvantage of running on Jython is that quite a few of the low-level test libraries within the RF ecosystem will not be available. Moreover, running in Jython will slap you with a performance penalty. Fortunately, in the mentioned situations, one could still run the stack on Python, through the usage of the so-called Remote Library Interface mechanism, that can be harnessed to connect the Python stack to an application and/or a test library running in a JVM (on the same or a remote system). We will be addressing the latter subject, as well, in one of our follow-up articles.

A possible, though highly simplified, set-up of an automation framework is the following:

Generic framework high-level design

Green signifies framework components whereas grey refers to components or artefacts, such as test code and product code, that are to be created by the development organization. The numbers indicate the order in which a typical test execution run would flow (more on this in the third post). The framework components are typical of all of today’s test automation frameworks. Obviously, this schema is a simplification of a real-life set-up, which would result in a more complex infrastructural model so as to account for topics such as:

• a possible distributed setup of the test engine and/or test driver
• parallel testing against a variety of interfaces (e.g. against REST and some UI) or against a multitude of product configurations/stacks/databases
• integration within a continuous delivery pipe line and with the test code repository
• etc.

Mapping these generic components onto concrete run-times within the RF stack, we get the following:

Robot Framework high-level design

The RF itself functions as the central framework engine. It is the core framework, that is being augmented by various tools and libraries that have been developed within the RF ecosystem, to form the larger, broader framework. (To be precise, in the given example, Selenium Webdriver does not belong to the RF ecosystem. But most of the other available low-level test libraries do.)

Let’s elaborate somewhat on the various components of the framework stack.

Test editor

The test editor is what we use to write, maintain and structure our automation code with. Test code not only consists of test cases, but also of various layers of abstractions, such as re-usable functions (keywords), wrappers, object-maps and global variables.

In the case of the RF, the editor can be anything, ranging from the simplest of text editors to a full-blown IDE. The Robot Framework comes with various editors, such as the RF Integrated Development Environment (RIDE), and with several plug-ins for popular IDE’s and text editors such as Eclipse, IntelliJ, Atom, TextMate or even Vim. But of course, you could also use a separate text editor, such as Notepad++. Which editor to use may depend on factors such as the required complexity of the test code, the layers to which one has to contribute (e.g. high-level test cases or re-usable, low-level test functions), the skill set of the involved automaton engineers (which may be business stakeholders, testers or developers) or simply personal taste.

Depending on the editor used, you may additionally benefit from features such as code completion, syntax highlighting, code extraction, test cases management and debugging tools.

Note that ‘official’ test runs are typically not initiated from within the editor, but through other mechanisms, such as build steps in a CI server or a cron job of some sort. Test runs are initiated from within the editor to test or debug the test code.

Test engine

The test engine, in our case the RF,  is the heart of the framework.

That is, it is the central component that regulates and coordinates and, as such, ties all components together. For instance, some of the tasks of the engine are:

• Parsing the test case files, e.g. removing white spaces, resolving variables and function calls and reading external files containing test data (such as multiple username/password pairs)
• Controlling the test driver (e.g. Selenium Webdriver)
• Catching and handling test library return values
• Error handling and recovery
• Aggregate logs and reports based on the results

Test driver

A test engine, such as the RF, is a generic framework and, as such, cannot itself drive a specific type of application interface, may it be UI (e.g. mobile or web) or non-UI (e.g. a SOAP service or an API). Otherwise it would not be generic. Consequently, to be able to drive the actual software under test, a separate layer is required that has the sole purpose of interacting with the SUT.

The test driver (in RF terms a ‘test library’ or ‘low-level test library’) is the instance that controls the SUT. The driver holds the actual intelligence to make calls to a specific (type of) interface. That interface may be non-UI (as would be the case with testing directly against a SOAP-service, http-server, REST-API or jdbc-database) or UI (as would be the case with testing against a web UI or Windows UI).

Examples of test drivers that are available to the RF are Selenium Webdriver (web UI), AutoIt (Windows UI) or the RF test libraries: Android library, Suds library (SOAP-services), SSH library, etc.

The latter are examples of ‘native’ RF test libraries, i.e. libraries that have been developed against the RF test library interface with the express intent of extending the RF. Some of these RF test libraries in turn re-use (that is, wrap) other open source components. The http library, for instance, reuses the Python ‘requests’ http client.

The former are existing tools, developed outside of the RF ecosystem, that have been incorporated into that ecosystem, by creating thin layers of integration code that make the external functionality available to the framework. Which brings us to the integration layer.

Integration layer

The main responsibility of the integration layer is to expose the functionality, as contained within an external tool or library, to the rest of the framework, mainly the engine and editor. Consequently,  the integration layer can also form a limiting factor.

Through the integration layer, the test code statements (as written in RFW syntax) are ‘translated’ into parameterized instructions that adhere to the syntax of the external tool. For instance, in the case of Selenium Webdriver, the RF integration library (called ‘Selenium2Library’) consists of a set of Python (module) files, that contain small functions that wrap one or more Webdriver functions. That is, these Python functions contain one or more Webdriver API-compliant calls, optionally embedded in control logic. Each of these Python functions is available within the framework, thus indirectly providing access to the functions as are exposed by the Webdriver API.

For example, the following function provides access to the Webdriver click() method (as available through the webelement interface):

[sourcecode language="python" collapse="false"]
def click_element(self, locator):
self._info("Clicking element ‘%s’." % locator)
self._element_find(locator, True, True).click()
[/sourcecode]

Within your editor (e.g. RIDE), the function ‘Click element’ can be used in your test code. The editor will indicate that an argument $locator is required.

These Python functions, then, are basically small wrappers and through them the integration layer, as a whole, wraps the external test driver.

As mentioned before, an integration layer is not necessarily part of the stack. Test drivers (test libraries) that have been written directly against the RF library API, do not require an integration library.

Our next post will elaborate on the keyword-driven approach to test automation that the RF follows.

Qxperts. We empower companies to deliver reliable & high-quality software. Any questions? We are here to help! www.qxperts.io