Automated UI testing: TestCafé

General
1

I wanted to ask if anyone has evaluated TestCafé for automated UI tests. It looks like it requires a lot less setup than the usual Protractor with Webdriver combo.
From what I could see:

  • Easier setup
  • Easier execution (incl. a Docker image with installed Chrome + Firefox)
  • Running Chrome + Firefox in headless mode
  • Typescript support
  • React support (via supplied React-Selectors)
2

From an A12 standpoint we do not have a deep understanding of TestCafé yet. In the course of [A12E-694]<INTERNAL_LINK> we had a brief look at different UI testing frameworks including also TestCafé.

From my personal standpoint I think that TestCafé seems to be really easy to use. Similar to Cypress without the limitation of only running in Chrome. But I also think that WebdriverIO and Nightwatch seems interesting.

However, keep in mind that for commercial use TestCafé require a license. The following text comes from the Licensing FAQ page.

I want to test web applications using TestCafe. What do I buy?

If you are using TestCafe to test “for profit” or “commercial” websites or applications, you must purchase a 12 month subscription from DevExpress or its authorized resellers for each user/tester. When you purchase a 12 month subscription to TestCafe, you receive 12 months of major and minor updates for each licensed product from the date of purchase.

Here the list of UI testing framework that we had a brief look with some notes:

  • Selenium
  • Appium (like Selenium but for mobile devices)
    • iOS 9.3+: Apple’s XCUITest
    • Before iOS 9.3: Apple’s UIAutomation
    • Android 4.2+: Google’s UiAutomator/UiAutomator2
    • Android 2.3+: Google’s Instrumentation. (Instrumentation support is provided by bundling a separate project, Selendroid)
    • Windows Phone: Microsoft’s WinAppDriver
  • Protractor (just wrapper for selenium)
    • Angular- Has special hooks, although can be successfully used with other JS frameworks too. Angular official documentation suggests using this tool.
    • Error reporting - Good mechanism.
    • Support - TypeScript support is available and the library is operated and maintained by the huge Angular team.
  • WebdriverIO (own implementation of Selenium)
    • Syntax - very easy and readable.
    • Flexible - A very simple and agnostic from even being used for tests, flexible and extensible library.
    • Community - It has good support and enthusiastic developer community that makes it reach with plugins and extensions.
  • Nightwatch (own implementation of Selenium)
    • Framework - Can be used with other frameworks too, but can be especially useful in case you want to run functional tests not as part of other framework.
    • Syntax - looks the easiest and the most readable.
    • Support - No typescript support and in general, this library seems to be slightly less supported than the others.
  • TestCafé (alternative to Selenium-Based tools)
    • Fast to set up - You don’t need special browsers. Just open a browser and run your tests there.
    • Cross Browser and Devices - Supports many browsers and devices and can be used with SauceLabs or BrowserStack that can provide the devices and the browsers to test on for you. This includes running of tests in Headless Chrome and Headless Firefox which will discussed later.
    • Parallel Testing - TestCafe can run your tests on several browser instances at once. This practice can significantly decrease your testing time.
    • Convenient Error Reporting
    • Own Ecosystem - TestCafe uses its own test structure. It could be quite convenient, especially because UI tests usually run separately from other tests, but some people dislike it.
  • Cypress (similar to TestCafe)
    • No Cross-Browser Support - Only support chrome right now (and not headless). They are working on it as this article is being created. (And we will update the article once they do it)
    • Lacks Advanced Features - Parallel testings and several testing tools are still missing comparing to TestCafe but they are on the road-map of the product’s diligent team.
    • Documentation - Solid and clear.
    • Debugging Tools - Easy debugging and logging of the test process.
    • Using Mocha as it’s test structure provider makes it’s use pretty standard and can make your UI tests be built in the same structure as the rest of your tests.
3

Thanks for the reply, it’s really helpful.

Regarding the license:

The documentation on this webpage describes TestCafe v2015.1

TestCafe itself is free to use now, only TestCafe Studio requires you to buy it :slight_smile: