Cypress vs TestCafé: Software Testing Tools Series Part 1

At Xebia we focus on building the right thing the right way. To do so we need to continuously receive feedback on the quality of our code. As such, a testframework that supports our way of working is paramount to success.

In this article we will have a look at Cypress and TestCafé CLI.

Allright. Let’s dive a bit into TestCafé and Cypress.

Run Tests, Run (CLI Magic)!

Cypress provides an epic Test Runner that gives you a visual structure of suites, tests, assertions, network requests, stubs, spies and more. The other part of the Test Runner displays the Application Under Test. When your test fails, the Test Runner will help you with providing information about why the test failed and giving you hints where to look.

To start the Test Runner, just start your testrun:

$(npm bin)/cypress run

The Test Runner will open in a new window.

To start an test run with TestCafé, just type:

testcafe chrome {path-to-testfile/}testfile

(instead of Chrome you can also use another browser like Firefox, Safari or Internet Explorer)
After starting TestCafé a new browser-window is opening, TestCafé is initializing in that browser and the test will run. After the test run, the browser will be closed and you will see the testresults on the commandline.

Debuggability

An awesome feature (understatement of the year) of the Cypress Test Runner is the possibility to go back to previous states of your Application Under Test. The previous state you select is a complete DOM snapshot of the Application Under Test at that time. You can use the development tools of the browser to inspect the source code. This makes debugging very easy.

In TestCafé, while debugging the Application Under Test, we set a .debug() in our code. After starting the testrun, the browser will open, the test will run and pause at the point where the debugger was set. There the Application Under Test can be inspected via the browsers’ Development Tools.

Live Reloading in Cypress and TestCafé

A very handy feature of Cypress is the live reloading capability.
This means that as you write your testscript and hit ‘save’, the Test Runner picks up the file and reruns the test. Even if this means breaking off the already running test. This gives you almost instant feedback on the test you are writing.

In TestCafe this is implemented a bit less intuitive.
When you edit and save the test file while your test is already running, you have to abort your test by ctrl-z the job (but then have to start TestCafé all over again) or you have to wait for the testrun to finish and then hit save again. So TestCafé listens to changes in the testfile only when the Runner is not running a test.

Selector Playground

A handy feature of the Cypress Test Runner is the Selector Playground.
You can use the Time Traveling feature to go to a specific step in your testrun and use the Selector Playground and choose an object. Cypress will return the best selector to use.

In TestCafé we set the .debug() in our testcode and run the test. When the the testrun pauses, we can use the browsers’ developer tools to inspect the code and find the best selector.

A best practice in using selectors can be found here .

Code Abstraction

Cypress is build on top of Mocha and Chai. Therefore you can use Mocha’s hooks (before(), after()) and Chai’s TDD and BDD assertion styles (like expect).

Below you can see a test written in TestCafé.

import { Selector } from 'testcafe';
fixture('Getting Started')
  t.page('https://devexpress.github.io/testcafe/example;')
test('My first test', async t => {
  await t
    .typeText('#developer-name', 'John Smith')
    .click('#submit-button')
    .expect(Selector('#article-header').innerText).eql('Thank you, John Smith!');
});

as you write your tests in TestCafé code, you have to take in account that you think of the asynchronity of Javascript. You have to ‘wait’ for the ‘promise’ (you could say the outcome) of the function.

The same test, but then in Cypress:

context('Actions', () => {
  beforeEach(() => {
    cy.visit('https://devexpress.github.io/testcafe/example')
  })
it('check Developer Name', () => {
  cy.get('#developer-name')
    .type('John Smith')
  cy.get('#submit-button')
    .click()
  cy.get('#article-header')
    .should('have.value', 'Thank you, John Smith!')
  });
})

here you see that Cypress has abstracted the async function execution away. A much cleaner coding style, if you ask me!

Browser Support

If you feel insecure about how your application will act in different browsers, the multi-browser support in TestCafé will be a big plus for you. TestCafé is able to run the tests in the following browsers (when installed on your system):

Google Chrome:
- Stable
- Beta
- Dev
- Canary
Internet Explorer (11+)
Microsoft Edge
Mozilla Firefox
Safari
Android browser
Safari mobile

Besides running the tests in the local browsers on a developers’ machine, TestCafé is able to run the tests headless in a pipeline and even on the cloud services like SauceLabs or Browserstack.

Cypress only supports the following Chrome based browsers:

Canary
Chrome
Chromium
Electron

Supporting more than this list is part of their roadmap and they are working on it as we speak, starting with FireFox. The same counts for SauceLabs and Browserstack support.

Wrapping Up: which UI Testing Tool will you pick?

Both Cypress and TestCafé are next generation test frameworks that will shorten your feedback loop and therefore shorten your lead times.
Here are my pro’s of both frameworks.

The pro’s of Cypress:

Easy debugabillity
Selector Playground
Great Documentation
Best implementation of Live Reloading

The pro of TestCafé

Browser Support in a box

To conclude this wrapping up, mainly because of its great debugabillity feature (the DOM snapshotting) and best implementation of Live Reloading, I would choose Cypress over TestCafé as E2E test framework.

I hope this post helps you finding your way in choosing the right framework for the right job. If you need further assistance in making the right choice or implementing the right framework or if I got something wrong or if you have any questions, feel free to contact me.

[update:]

read part 2 here: https://xebia.com/blog/cypress-and-testcafe-a-comparison-part-two/

Thanks for reading 🙂

I am a specialist at Qxperts. We empower companies to deliver reliable & high-quality software. Any questions? We are here to help! www.qxperts.io