The open source Robot Framework (RF) is a generic, keyword- and data-driven test automation framework for acceptance test driven development (ATDD). As such it stands alongside similar, but more well-known frameworks, like FitNesse, Cucumber, et alia. The (relative) unfamiliarity of the testing community with the RF is undeserved, since the RF facilitates powerful and yet simple test automation against a variety of interfaces and features some distinct advantages when compared to those other frameworks.
In a series of blogposts, we would like to make a case for the Robot Framework, by showing its greatness through a number of hands-on examples from my upcoming workshop. Next to demonstrating its advantages and strengths we will also expose some of its drawbacks and limitations, as well as touch upon certain risks that flow from harnessing some of its unique features.
Our first three posts will give an introductory overview of the RF, laying the conceptual foundation for the remainder of the series. Therefore these three articles will not concentrate on practical, hands-on examples or instructions, but instead have a more theoretical feel. Moreover, several of the fundamental concepts laid out in them, are applicable not only to the RF, but to most (if not all) test automation frameworks. Consequently, these first three posts target those that miss a basic understanding of test automation (frameworks) in general and/or of the RF in particular. The remainder of the series will be also of interest to more seasoned automation engineers.
We will first look into the basic architecture that underlies the framework and discuss the various components that it is being comprised of. In the second post we will discuss the nature of the keyword-driven approach that the RF entails. The third post will detail a typical test automation work flow.
For a first-hand experience of the pros and cons of the RF, you might want to join the first Robot Framework meetup in the Netherlands.
Robot Framework stack
The RF is written in Python. Consequently, it runs on Python, Jython or IronPython. The framework can thus be used with any OS that is able to run any of these interpreters, e.g. Windows, Linux or OS X. Unless you have a specific reason to do otherwise, the RF should run on Python. A typical situation that would require e.g. Jython, would be automating against a Java application or implementing your own RF test library in Java (more on this in a later post). A disadvantage of running on Jython is that quite a few of the low-level test libraries within the RF ecosystem will not be available. Moreover, running in Jython will slap you with a performance penalty. Fortunately, in the mentioned situations, one could still run the stack on Python, through the usage of the so-called Remote Library Interface mechanism, that can be harnessed to connect the Python stack to an application and/or a test library running in a JVM (on the same or a remote system). We will be addressing the latter subject, as well, in one of our follow-up articles.
A possible, though highly simplified, set-up of an automation framework is the following:
Green signifies framework components whereas grey refers to components or artefacts, such as test code and product code, that are to be created by the development organization. The numbers indicate the order in which a typical test execution run would flow (more on this in the third post). The framework components are typical of all of today’s test automation frameworks. Obviously, this schema is a simplification of a real-life set-up, which would result in a more complex infrastructural model so as to account for topics such as:
- a possible distributed setup of the test engine and/or test driver
- parallel testing against a variety of interfaces (e.g. against REST and some UI) or against a multitude of product configurations/stacks/databases
- integration within a continuous delivery pipe line and with the test code repository
- etc.
Mapping these generic components onto concrete run-times within the RF stack, we get the following:
The RF itself functions as the central framework engine. It is the core framework, that is being augmented by various tools and libraries that have been developed within the RF ecosystem, to form the larger, broader framework. (To be precise, in the given example, Selenium Webdriver does not belong to the RF ecosystem. But most of the other available low-level test libraries do.)
Let’s elaborate somewhat on the various components of the framework stack.
Test editor
The test editor is what we use to write, maintain and structure our automation code with. Test code not only consists of test cases, but also of various layers of abstractions, such as re-usable functions (keywords), wrappers, object-maps and global variables.
In the case of the RF, the editor can be anything, ranging from the simplest of text editors to a full-blown IDE. The Robot Framework comes with various editors, such as the RF Integrated Development Environment (RIDE), and with several plug-ins for popular IDE’s and text editors such as Eclipse, IntelliJ, Atom, TextMate or even Vim. But of course, you could also use a separate text editor, such as Notepad++. Which editor to use may depend on factors such as the required complexity of the test code, the layers to which one has to contribute (e.g. high-level test cases or re-usable, low-level test functions), the skill set of the involved automaton engineers (which may be business stakeholders, testers or developers) or simply personal taste.
Depending on the editor used, you may additionally benefit from features such as code completion, syntax highlighting, code extraction, test cases management and debugging tools.
Note that ‘official’ test runs are typically not initiated from within the editor, but through other mechanisms, such as build steps in a CI server or a cron job of some sort. Test runs are initiated from within the editor to test or debug the test code.
Test engine
The test engine, in our case the RF, is the heart of the framework.
That is, it is the central component that regulates and coordinates and, as such, ties all components together. For instance, some of the tasks of the engine are:
- Parsing the test case files, e.g. removing white spaces, resolving variables and function calls and reading external files containing test data (such as multiple username/password pairs)
- Controlling the test driver (e.g. Selenium Webdriver)
- Catching and handling test library return values
- Error handling and recovery
- Aggregate logs and reports based on the results
Test driver
A test engine, such as the RF, is a generic framework and, as such, cannot itself drive a specific type of application interface, may it be UI (e.g. mobile or web) or non-UI (e.g. a SOAP service or an API). Otherwise it would not be generic. Consequently, to be able to drive the actual software under test, a separate layer is required that has the sole purpose of interacting with the SUT.
The test driver (in RF terms a ‘test library’ or ‘low-level test library’) is the instance that controls the SUT. The driver holds the actual intelligence to make calls to a specific (type of) interface. That interface may be non-UI (as would be the case with testing directly against a SOAP-service, http-server, REST-API or jdbc-database) or UI (as would be the case with testing against a web UI or Windows UI).
Examples of test drivers that are available to the RF are Selenium Webdriver (web UI), AutoIt (Windows UI) or the RF test libraries: Android library, Suds library (SOAP-services), SSH library, etc.
The latter are examples of ‘native’ RF test libraries, i.e. libraries that have been developed against the RF test library interface with the express intent of extending the RF. Some of these RF test libraries in turn re-use (that is, wrap) other open source components. The http library, for instance, reuses the Python ‘requests’ http client.
The former are existing tools, developed outside of the RF ecosystem, that have been incorporated into that ecosystem, by creating thin layers of integration code that make the external functionality available to the framework. Which brings us to the integration layer.
Integration layer
The main responsibility of the integration layer is to expose the functionality, as contained within an external tool or library, to the rest of the framework, mainly the engine and editor. Consequently, the integration layer can also form a limiting factor.
Through the integration layer, the test code statements (as written in RFW syntax) are ‘translated’ into parameterized instructions that adhere to the syntax of the external tool. For instance, in the case of Selenium Webdriver, the RF integration library (called ‘Selenium2Library’) consists of a set of Python (module) files, that contain small functions that wrap one or more Webdriver functions. That is, these Python functions contain one or more Webdriver API-compliant calls, optionally embedded in control logic. Each of these Python functions is available within the framework, thus indirectly providing access to the functions as are exposed by the Webdriver API.
For example, the following function provides access to the Webdriver click() method (as available through the webelement interface):
[sourcecode language=”python” collapse=”false”]
def click_element(self, locator):
self._info(“Clicking element ‘%s’.” % locator)
self._element_find(locator, True, True).click()
[/sourcecode]
Within your editor (e.g. RIDE), the function ‘Click element’ can be used in your test code. The editor will indicate that an argument $locator is required.
These Python functions, then, are basically small wrappers and through them the integration layer, as a whole, wraps the external test driver.
As mentioned before, an integration layer is not necessarily part of the stack. Test drivers (test libraries) that have been written directly against the RF library API, do not require an integration library.
Our next post will elaborate on the keyword-driven approach to test automation that the RF follows.
Qxperts. We empower companies to deliver reliable & high-quality software. Any questions? We are here to help! www.qxperts.io
I definitely agree with you, RF is a powerful tool that is too often overlooked. RIDE is an easy way for members of my team that aren’t coders or programmers, to automate tests. RF integrates well with Jenkins (formerly Hudson) which can start the tests and track results.
The combination of Selenium and AutoIT libraries give the ability to control dialogue boxes in a web browser such as load and save that you can’t do with standalone Selenium.
I have been using it for around 10 months and see no reason to switch to anything else.
Hi Tony,
Thank you for your comment.
It is the combination of the powerful (and yet easy-to-grasp) RF scripting facilities on the one hand and the IDE functions of RIDE on the other hand, what makes Robot Framework such a powerful tool in the hands of non-coders also. It empowers non-technical members of the team, by making them less dependent on programmers to write mid-level test functions. Being able to create these functions themselves, they have much more control over, what Gojko Adzic calls, the User interface workflow level and Technical Activity Level. With other generic testing frameworks, most of the test case flows on these levels are written (hard-coded, so to speak) in low-level code (e.g. the Java fixtures in FitNesse). Of course, you also introduce a maintainability risk. The latter can be mitigated (i.a.) through the application of best practices. More on the latter in a follow-up post.
We will be doing posts on RIDE and Jenkins integration as well.
RF is good what was designed for: ATDD.But this test phase is the final one. What happens with functional (module), system and integration tests? RF is a nightmare in these phases . I think that RF should remain a very niched tool
Hello Dan,
Thank you for your reply.
ATDD is not a test type or test phase. It is a technique or method that supports an agile approach to software development. That is, it enables fundamental aspects of such an approach, such as collaboration (e.g. wrt the specifications) and being test-driven. So one should not mistake ATDD for the more traditional concept of an acceptance test type that is being executed at the end of a phased trajectory, as some sort of validation of the ‘final exit criteria’ for the whole process.
The RF has been designed to support ATDD. But it is, in my opinion, certainly not limited to such a context. RF is, foremost, a keyword-driven framework. As I have tried to explain in my second post, calling a framework an ATDD framework, means, foremost, using it in a specific way. But it can be used in a non-ATDD context as well. And even in a non-agile context. The context determines the way a framework should be used in order to contribute meaningful test automation within that context.
In terms of test types that can be automated (regardless the context in with these test types are applied), then, I believe the RF supports various test types, such as acceptance testing, system testing and regression testing. And even integration testing. Some of my intended follow-up posts will go into this topic of different test types and the possible role of the RF. One of the unique strengths of the RF is it’s support, out of the box, of a broad range of interfaces and programming languages. As such it is pretty easy to hook into any layer of a product and thus to conduct tests that target isolated components, end-to-end system tests or anything in between.
Finally, I am wondering what you mean with a ‘nightmare’? Can you elaborate somewhat on that? And are there frameworks that you would recommend in the situations you experienced the RF to be a nightmare?
Thanks again for your input,
Michael
Hi,
I read quite a lot about RF, and work with it for some time. As for me RF is good only when we speak about non-programming skilled testers, otherwise it’s not so sexy as u write here.
RIDE – is just hell, it even not provide a features to Find and Replace words !! When you press save – all you focus in text editor canceled. Accidentally press run when none of tests selected – its run all the tests (maybe its feature, but stop button doesn’t work properly too in this case).
What about debug ? Use Sleep and Logs is just for people from dinosaur era.
It’s just not so user-friendly after using it for while
PS. it’s only my point of view :_)
Hi Igor,
Thanks for your comment. By the way, I deleted one pejorative. I hope you don’t mind; I am kind of old-fashioned. 🙂
The RF definitely (and demonstrable) is not ‘good only’ when used by non-technical testers. Quite the contrary, since (amongst other advantages) it fully supports easy and flexible creation of lower-level fixtures in code, by means of it’s simple and powerful test library API.
Come to think of it, the RF actually is super-hot for techies. This very afternoon I gave a RF demo to an architect. A hardcore techie, who also knows his test automation. And I (that is the RF) impressed him greatly. As a matter of fact, I see this happen all the time: especially programmers, in my experience, appreciate the combination of power, versatility/flexibility, simplicity, performance and elegance that the RF has to offer.
You see, the scripting facility (that you are probably referring to when talking about being good only for non-coders) is an extra. The scripting, admittedly, is something that specifically non-technical testers would employ, to create for instance small and simple wrappers (e.g. to test broader flows throughout the chain), setup/teardown routines or (non-essential) convenience functions. The advantages of such basic levels of scripting are (a.o.) that it empowers non-technical testers and makes them less dependent on help from developers for small tasks at hand. It also serves as a stepping stone to learning how to code and motivates testers to further themselves in this respect. Of course there are also risks involved when giving such power into the hands of inexperienced/beginning coders. Foremost maintainability and stability risks, but also a potential vendor/tool lock-in. As I have written elsewhere, various effective counter-measures exist to mitigate those risks.
The powerful scripting facility is something that is unique to the RF. However, for our discussion, the important thing to understand is that scripting can, but most certainly does not have to, be employed. Lower-level test functions can be written directly in code, just like with other frameworks, completely bypassing the scripting. So, you can use it, given a certain context calls for it and given a healthy approach (not too much, not to complex, apply design patters and other best practices to the script code, etc.). But you don’t have to! You can create a full-fledged automation solution without having to ‘resort’ to RF scripting at all.
I moreover suspect that you are confusing RIDE (one of many possible editors for RF) with the RF itself. This happens quite a lot, but: RIDE is not RF! Not at all. RIDE has indeed been developed as a Robot Framework IDE for beginning automators/coders. And as such it works wonders, as I have experienced quite often in the field. This is due to the fact that it comes with some truly wonderful features as well, such as lots of refactoring functions. It helps testers understand and apply all sorts of design patterns, without sporting the intimidating complexity of (the interface of) Eclipse or somesuch IDE. It also gets them into a very steep learning curve: just like the scripting, it is an ideal stepping stone to learning how to properly design and write test code.
Sure, RIDE is not bug free and it still misses some (important) features. But it is very actively developed and all of the stuff you mentioned is being planned for in future releases (as can be gathered from its GitHub pages and issue tracker).
And, again, RIDE is just one of many possible editors. There are plug-ins (adding support for stuff like syntax highlighting, auto-complete, etc.) for a myriad of editors:
IntelliJ, Eclipse, Sublime, Atom, Brackets, Vim, Notepad++, Emacs, Gedit, TextMate, Red.
So make your pick!
But don’t write off Robot Framework because you don’t like an editor. It would be your loss! 😉
Thanks again for your reply.
Regards,
Michael
PS: The need to find and replace words may be an indication of a missing abstraction layer.
May i Know , what benifits of Robotframework in selenium and QTP.
And what are the shortcomings.??
Thank you for your reply.
Unfortunately, it is not entirely clear to me what the actual questions are. So I hope that I will be providing you with some relevant answers. 🙂
Comparisons between frameworks, especially in terms of benefits and drawbacks, take up whole blog posts (at least when done well). Moreover, what feature or lack thereof may be considered to be a benefit or drawback and/or what the impact thereof might be, depends on quite a few contextual factors. Not taking these factors into account, constitutes a dangerous, tool-centric approach!
Having said that, let’s take a brief and superficial look.
RF and QTP (the latter is nowadays known under the name of ‘Unified Functional Testing’ or ‘UFT’) are similar frameworks, in the sense that both provide you with a generic framework as well as some low-level libraries to drive various automation interfaces. The generic framework (or engine) provides you, in both cases, with a very broad range of test automation functions, such as error handling and recovery, reporting and lots and lot’s of other such ‘convenience’ functions. They also both integrate with low-level libraries to drive automation interfaces such as a web gui, a db or a SOAP service.
So. in a sense they are competing frameworks, both targeting enterprise level automation projects.
Of course there are differences. Let’s mention a few:
UFT is a commercial product, RF is an open source framework.
UFT integrates, out-of-the-box, with ALM, RF does offer very limited test management capabilities.
RF comes with support for a very broad range of technologies, i.e. automation interfaces, such as web, http, mobile (Android, iOS), SOAP, jdbc, odbc, JMS or even network protocols. UFT is significantly more limited in this respect. However, UFT supports several technologies that may be of value to certain enterprise environments, such as BlueZone (a terminal emulator for IBM main frames), Siebel, SAP, etc.
RF supports a broad range of languages to write your automation code in, such as Python, Java, ,NET, Ruby, PHP, Perl and others (admittedly, most of them through the so-called remote library interface mechanism). UFT supports merely VBScript.
RF fully supports the keyword-driven approach to test automation, out-of-the-box. UFT only through writing a platform extension, that will bind the framework to an external tool such as Excel. only through such a shell around UFT, will you be able to write keyword-driven test cases.
There are lot’s of other differences and similarities that are worth mentioning, but people are complaining about the length of my replies as it is already. So I will leave it at that. 🙂
Finally, on the relation between RF/UFT and Selenium Webdriver:
There is no sense in comparing RF and Selenium Webdriver (or UFT and Webdriver). RF and UFT both are generic test automation frameworks. Selenium is an API that you may write test code (functions and test cases) against to automate your tests against a web gui. RF can wrap this API, so that you may write the code in/on the framework instead of writing it directly in code (e.g. writing Selenium test code in Java, using e.g. Eclipse and JUnit). In doing so, you gain access to all of the framework convenience functions, which you won’t get when writing your tests outside of the RF or a similar generic test automation platform.
As far as I know, UFT currently does not reuse Webdriver, but, rather, uses its own, proprietary web fixture. But as I understand it, an upcoming release of UFT will support Webdriver as well.
We use Testlink Reguirement management/ Test management together with RF. All RF tests are imported in Testlink and the RF listener reports the result of every testcase/ testsuite of every build automaticly to the test management program.
In my opinion the only downside of RF is the limited test result history. The only options are either html or xml files. By using a test management program (like test link) everything is automaticly put in a database, which make it a lot more easier to track your history.
Hi,
Thanks for your input.
Yes, you are absolutely right.
As I just wrote in my previous reply (to the previous comment), open source frameworks such as RF often do not provide (extensive) test management features (compared to commercial end-to-end frameworks, such as Tosca or UFT).
RF indeed simply writes the test run results into a variety of output files, supporting just a few formats (such as xml). These formats, of course, are highly portable. So there should be plenty tools and frameworks that the test run results history can be imported into (either directly or with a prior conversion step).
Regards,
Michael
Hi,
You mentioned about importing the RF tests into TestLink. Could you please provide the details/flow on how to do it?
Your response will be highly appreciated. Thank you!
I am looking to automate a myriad of websites using a robust and user-friendly framework. What I am looking for basically is to:
a) Avoid rewriting the same code for various websites
b) Build common object repositories / test functions which I can use across all webpages.
c) Good compatibility with external tools such as Jenkins for execution tracking, CI and result reporting
From what I have read, RF seems to be all that and more. I have to give a proposal to my company’s higher management for a good tool and I am thinking of RF. So will RF qualify as a good tool?
Apologies for my lack of knowledge of RF, but I thought putting forth this question to an expert was the best way rather than read the entire documentation online.
Hi Srinivas,
First of all, apologies for my late reply.
I would assume that, in the meanwhile, your question either has been answered or is no longer relevant. But maybe there are other people having the same questions.
So: yes, the RF is all that and more!
I won’t go into details though, since in the various blog posts that I have published until now, I have already described the various advantages that make the RF ideal in your case, given the challenges you describe. From these posts it becomes apparent that the RF optimally facilitates and even stimulates the application of various design patterns that will solve the problems at hand and fulfill the company’s needs.
Of course, it must be pointed out (like I did in the blog posts) that it is not really the tool/framework, but rather your skill and creativity as an automator that will have to solve the problems and fulfill the needs. But the tool can make it more or less easy for you. For instance, it can slow down or speed up the flow of creativity. However, in the case of the RF, you get optimal support. Harnessing its power and versatility (as described in the posts), you can create maximum reusability (and maintainability) of your domain functions, object mappings and other low-level resources.
But YOU need to identify and the implement the proper abstraction layers and apply all of the other relevant design patters. The RF facilitates creating and subsequently using the resources in those layers.
As to the compatibility issue: the RF integrates perfectly within Jenkins. The reporting facilities are good out-of-the-box and are also extensible in a very easy and efficient manner.
Regards,
Michael
Yes I admit that RF facilitates lot of things that the people who use traditional way don’t understand its depth, I’ve been into many automation tools and technologies but RF is the best in most aspects. I’ve worked on RF ~2.5 yrs with Python and Java it allows the command line options like pybot kinda which provides good flexibility when u go for CI tools like Jenkins and ofcourse locally. You can also use RF for DD, KD and hybrid framework test development.
Hi Munindar,
Thanks for your comment. I have nothing to add. 🙂
Michael
Even I’ve scripted automation tests for AS400 IBM Mainframes application using PCOM emulator using Python bindings.
Hi Munindar, Can you please provide library files that you might have developed for automating AS/400 and used it in RobotFramework?
I am new to the network automation using python…..May i know how exactly RF is useful and how and where should i initiate to learn that from the first step
Hi Purushotham,
Well, for answers to both questions, you could read my various posts and, of course, consult the abundance of RF documentation and other resources (such as RF user groups and Slack) available on the web.
Start here for your first overview of available guides and other documentation:
http://robotframework.org/#documentation
Regards,
Michael
Hi Michael, I am a newbie for RF Framework but I have been working on Selenium for the past so many years using various different frameworks, including Customized Cucumber and my own coded framework using Java. I am trying to learn the advantages of RF. Definitely there are lot of advantages in RF for non-coders, but I want to understand what is the scope of Automation team members who love coding (like me, to some extent 🙂 ). Would it be simply using the RIDE (one of the IDE’s), go through UI steps to create new TC’s or something that can be CODED. Also, it will be of great help if you could make me understand how different it is from Cucumber Framework.
P.S: I am trying to figure out framework for a completely new product. Hence, exploring the options available. Apologies for lengthy question but at least I see a similarity between us (in writing lengthy posts). 🙂
Hi Subhash,
What do you mean, ‘lengthy posts’?! I don’t write lengthy posts!! 😉
So. let me provide you with some of my typically concise answers. 🙂
“[…] what is the scope of Automation team members who love coding (like me, to some extent)[?]”
As I have written elsewhere:
“The powerful scripting facility is something that is unique to the RF. However, for our discussion, the important thing to understand is that scripting can, but most certainly does not have to, be employed. Lower-level test functions can be written directly in code, just like with other frameworks, completely bypassing the scripting. So, you can use it, given a certain context calls for it and given a healthy approach (not too much, not to complex, apply design patters and other best practices to the script code, etc.). But you don’t have to! You can create a full-fledged automation solution without having to ‘resort’ to RF scripting at all.”
So you can use the proprietary RF scripting language to code reusable test functions (keywords), but you can also write these functions directly in code. For coding you can use various languages, such as Python, Java, .Net, Ruby, et alia (see my post on the Remote Library Interface or the official RF documentation for an overview of all supported languages). With RF it extremely simple to create libraries of low-level domain functions in code!
On the questions … :
– When and why to use coding and when and why to use scripting (or when and why to use a combination thereof).
– The advantages, disadvantages and risks of using coding and of using scripting (which, of course, has impact on the previous question).
– How to write domain functions (keywords) in code.
… I plan to write a separate blog. So I do not want to go into these here.
The above applies to automation with Selenium Webdriver as well. That is, you could script domain functions by using the keywords in RF’s Selenium2Library, where needed embedding the keyword calls in the control logic as offered by the various RF scripting constructs (such as FOR loops, conditionals and the like). And/or you could write domain functions directly in code, programming in e.g. Java or Python against the Webdriver API. You could group logically related functions in your own Java/Python/etc. test libraries. The keywords in these libraries can then be used in the same way in the RF test cases as the functions that were created in scripting. You can use both alongside each other in the same test case (or even statement) and it would be fully transparent to the test case author that one function has been scripted and the other has been coded.
“Would it be simply […], go through UI steps to create new TC’s or something that can be CODED.”
BTW: depending on the test object and the automation interfaces available, you should preferably automate against a service/api layer.
“Also, it will be of great help if you could make me understand how different it is from Cucumber Framework.”
Well, I really would like to be brief on this, as an effective framework comparison is quite laborious. So, to simplify matters, Cucumber is mainly a collaboration tool, used in an ATDD/BDD context. RF is a ‘hardcore’ test tool, on top of being able to be used in an ATDD/BDD context (as it, for instance, also has support for Gherkin). RF as a generic testing framework is very, very powerful, flexible and versatile (see my other posts on the RF). Demonstrable more so than Cucumber or any other, similar framework for that matter. It’s ecosystem of low level test libraries is also unique in its richness.
But which framework to use, very much depends on what the context of your automation effort is, in terms of (amongst others) its high-level goals and scope (I have written about this elsewhere in more detail).
Regards,
Michael
Hi, is there a way to re-run failed test cases as and when they fail in Robot?
No, I dont want the –rerunfailed keyword, it only works on a suite level after the entire execution is done, I’m trying to run the failed testcases as and when they fail.
Tried re-running them using Listeners::end_test(). But no use. Though the re-run happens, it is not anyway close to how it should work?
Is this do-able in this so-called “GREAT” framework? If yes, how should it be done?
Hi Krishna,
Thanks for your question.
I myself have not yet encountered a situation in which such a mechanism was required. So, apart from the –rerunfailed option and the listeners, I have no clue. 🙂
Execpt maybe a crude work-around using the ‘Run keyword if test failed’ keyword, that you could employ as a defaut test teardown at the highest suite level. The keyword to run you would then have to write yourself and would have to contain some small and simple logic to re-run a current test case upon failure. This should be easy to do.
BTW:
“Though the re-run happens, it is not anyway close to how it should work?”
Could you then please specify how it should work in your opinion?
Regards,
Michael
And second, the only way you can run your test-cases in an explicit order is by adding numerical conventions to that file or folder. Seriously, how bad is this??
If not that u need to either separate all your test cases and run them individually. They talk about “argument files” but as far as I see, it does not help at all.
It is pathetic that the command, robot –test “testcasetoberunfirst” –test “testcasetoberunsecond” –test “testcasetoberunNth” has to be all hand-written in a seperate feature which is called “ARGUMENT FILE”. Seriously, lame.
Hi again,
There are indeed a number of ways in which to deviate from the default execution sequentiality. Some of which you mention. All of which require the user to specify the order ‘hard coded’ or through a rule set.
I think the important question here is, why you need to specify the order of execution. As a matter of principle, you should be able to run any random subset of your test set (i.e. one, some or all test cases or suites) and be able to do so in any order. The ability to do so, is called ‘test case independence’. I will not go into the how and why of test case independence here, but suffice it to say that there should be a good reason to deviate from this principle.
Regards,
Michael
Hi Michael,
I have used robot framework for web UI automation in my last experience and found it awesome. Some of my queries are:
1.) Any support email ids. To raise technical concerns.
2.) I can add a python function as keyword, But suppose that function calls internally 5 more functions. Will the logs.html can show where does it actually failed ?
3.) Is there any debugging feature provided by any Id for Robot framework (i need for web automation only)
Hi,
Thanks for the feedback. Glad you like the framework. Then again, what’s not to like. 😉
As to your questions:
1. Please see: http://robotframework.org/#support
2. Yes, it most certainly will. It doesn’t matter whether you call a keyword from an external test library (such as Webdriver) or from your own keyword set (whether it be written in code or in script). The log will always show the keyword that raised or led to the error/failure.
3. For Selenium2Library specifically, see the ‘Debugging Selenium2Library’ section on https://github.com/robotframework/Selenium2Library/blob/master/BUILD.rst. In general, see: http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#using-the-python-debugger-pdb. There is also a debugging library for RF (updated today): https://github.com/xyb/robotframework-debuglibrary.
4. The latest version of RF supports Python 3. Do you have any specific problems or questions regarding RF with Python 3?
Thanks again for your questions!
Michael
Hi Michael,
On 4: I meant for selenium2Library, when is the plan to support Python 3.
One more question: Like Robot has doc page for its libraries (ex: http://robotframework.org/Selenium2Library/Selenium2Library.html#Page%20Should%20Contain).
Suppose i created my own library, is there any tool which will generate docs for my library (can be a html file) ?
Hi,
I apologize for my late reply. Somehow, I overlooked this last question of yours.
Thus, you might have found answers to your questions in the meanwhile. However, for other people that might come here searching for answers to these questions, I would like to reply nevertheless.
1. As of 01-09-2017, the Selenium2Library has been redesigned and now also supports Python 3. See: https://pypi.python.org/pypi/robotframework-selenium2library.
2. Please see: http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#library-documentation-tool-libdoc. I have used this tool myself and it works great.
Regards,
Michael
4.) How to use RF with python3 ?
Hi,
Please see my answers to your previous comment.
Michael
I like the reporting part of robot framework but creating test cases is double work. You write the code in python and then you figure how to write the robot part. I would much prefer it if was with out the sentence part and you could just plug in your code.
In practice, you need scripting skills all the time.
Hi Michael,
Thanks for your comments.
However, I do not quite get your point.
First of all, no matter which platform/framework you will use, you will always apply layering. And at the highest abstraction level, you will always separate your test designs from the reusable, lower-level test functions (and other such artifacts). I have written in my blog posts about how and why to do this (as have lots of other people of course). Moreover, it doesn’t matter (in THIS regard, that is) whether you would write the lower-level stuff in code (Python, Java, etc) or in RF script. (Of course, in other regards it matters a LOT).
Secondly, you CAN write your functions etc in Python (or other languages) OR in script OR in a combination thereof (which option you choose, is context-dependent).
Thirdly, you write your test cases in simple natural language sentences. They should not contain code, whether it be Python (or whichever language you use) or RF script. Again, this should be case regardless the framework you use (e.g. Cucumber).
Regards,
Michael
Hi,
Can we use RF framework across different applications
iOS, Android
Windows: Win32, UWP
Firmware
Client Server
Cloud
I am thinking of a common framework to use across different applications
Thanks,
Kiran
Hi Kiran,
Thanks for the questions.
RF supports iOS and Android through several external test libraries: http://robotframework.org/#libraries.
I do not know what you mean by ‘Windows – Win32’: RF can run on Win32/64. It can test against all kinds of software that is developed for and/or runs on Win32/64. It can test against the Win-GUI through an external library.
Similarly, wrt ‘Windows – UWP’: do you mean whether RF can test apps developed against the UWP API and/or running in/on the Windows Runtime platform?
With ‘Firmware’ you mean testing embedded software?
Client/server and Cloud: these are more system topologies/architectures. Not sure what you are asking here.
Regards,
Michael
let me know steps of install extent report in Ride tool(robot framework)
any suggestion about robotframework-httplibrary for python3.x . since it is not supported is there any alternative? thanks in advance
Hi Shalva,
You can use the RF RequestsLibrary. That uses the Python HTTP client requests internally. The project is very active. The lib runs on Python 3.
Furthermore, there is an HTTP library that is based on the (Java) Apache HTTP client.
Regards,
Michael