Skip to content

The time for UI regression testing is now!

A quick personal story on why it hasn’t been time for UI regression testing before, and what changed?

But first: User interface regression testing, what is it?

Showing an ok-button and a mouse pointer about to click it. Then some pseudo-code for testing the UI of the button.

To automatically test your user interface to see that it still works and haven’t fallen apart since last you wrote some code or did some changes.

Quick and simple. It’s to check if the user can still achieve what she or he could last time you changed your code. It’s close to testing that the intentions of your code and interfaces are still working.

My story

This is my personal story on UI regression testing, so first I need to tell quickly where I come from.

A person (me) with a sun and a mobile app + a moon and some code.

I’m an interaction designer by occupation, and on some of my spare time I program stuff. I’m not a very good programmer, but I have fun. And I have big plans.

A search box with an active cursor.

Mostly what I do is creating modules and stuff that can be used for small search engines.

12 magnifying glasses

I want to make it possible to create a lot of search engines.

A lot of magnifying glasses, more than I want to count.

And I mean a lot. For anyone to create and use. Easily, and at almost no cost.

Node.js-logo, GitHub cat and Travis CI-person.

I started with Node.js. It was easy to get going. Always a library at hand when I didn’t manage to fix stuff my self. Then I started using GitHub to store my code, and Travis CI to make sure what I had achieved continued working.

A mobile phone with some sort of an app. Green backround color on app.

After a while I created small functional demos in HTML, CSS and JavaScript. To test my ideas and see how I could create better demos and get closer to my goal.

Phone app to the left with an arrow and a clock pointing to another phone app on the right. The one on the right displays an error message and has a pink background.

But since I didn’t have any good ways of testing the UI, each and every one of the demos ended up breaking without me knowing when it happened. Somewhere along the way I had changed some code, or some dependency had changed, and that was it.

Selenium logo

Which meant I needed automated tests for the demos. On the user interface level. First framework I heard about was Selenium 10 years ago on a project I worked on for a customer. It seemed really promising.

Selenium logo with a screwdriver and a monkey wrench below it.

But one big problem I remember really well: Mostly the developers worked on fixing Selenium rather than fixing the actual code. It wasn’t stable enough.

PhantomJS logo

Then I started reading about PhantomJS. Based on Webkit (Same as Safari). You could control the browser with JavaScript and by that test the UI with it, but it was lacking a lot of framework for me to actually get things done.

PhantomJS-ghost outrun by Safari/webkit compass.

And before I got to test PhantomJS it was outrun by updates to WebKit. PhantomJS had a small group of developers and keeping up with changes in the upstream WebKit was too much to handle. Today the project is abandoned. with text daydream/nightmare below.

Then I found Daydream / Nightmare created by Segmentio. Daydream could record interactions you did in the browser and you could re-play it as automated tests in Nightmare which was an Electron-based browser (which was based on Chromium). outrun by Chromium logo.

But same problem with Daydream & Nightmare. Too little organisation to keep up with upstream updates. Chromium/Electron pulling ahead, and de facto abandoned the project.

It seemed to me that every time I found something interesting, it was already about to die.

But what about Playwright?


Then, a little over two years ago, I found Playwright and started to read about the framework/project. It seemed quite promising, and I wanted to have a go. It's supported by Microsoft — so it’s more likely it will be kept alive and up to date.

Playwright logo. Interaction record possibility. CSS selectors & xpath. Test-runner. Logos of Safari, Chrome and Firefox. A laptop and a phone.

The features was just what I needed:

  • Recording of UI interactions
  • CSS selectors for interacting with the page. xpath if that’s your choise.
  • Test-runner of your choise, mine is Ava.js.
  • Supports Webkit (Safari), Chromium (Chrome) and Firefox browsers.
  • Can emulate smartphones in addition to regular web pages.
  • Runs on Linux, OSX and Windows.

I started experimenting with Playwright around 10 months ago and now all but one library with HTML demos have UI regression tests. And I get a breaking test and a notification if something goes bust. Yay!

A question written, saying "Is the time now?" a person with a T-shirt saying "You can do it!" that same person saying "Yes!".

So, if you're working on a web project and you have some user interface, and you're not doing UI regression testing, I would hope that you at least took a quick look at

Disclaimer: From what I hear, Selenium is alive and kicking and doing great, so don’t take my word for it if you wonder if it’s a good choice today. But it wasn’t a  great choice 10 years ago. And I guess there weren’t many options either =)


Did you like this post? Read my post about Test setup for JavaScript/web  development with batr here.