A quick personal story on why it hasn’t been time for UI regression testing before, and what changed?
But first: User interface regression testing, what is it?
To automatically test your user interface to see that it still works and haven’t fallen apart since last you wrote some code or did some changes.
Quick and simple. It’s to check if the user can still achieve what she or he could last time you changed your code. It’s close to testing that the intentions of your code and interfaces are still working.
My story
This is my personal story on UI regression testing, so first I need to tell quickly where I come from.
I’m an interaction designer by occupation, and on some of my spare time I program stuff. I’m not a very good programmer, but I have fun. And I have big plans.
Mostly what I do is creating modules and stuff that can be used for small search engines.
I want to make it possible to create a lot of search engines.
And I mean a lot. For anyone to create and use. Easily, and at almost no cost.
I started with Node.js. It was easy to get going. Always a library at hand when I didn’t manage to fix stuff my self. Then I started using GitHub to store my code, and Travis CI to make sure what I had achieved continued working.
After a while I created small functional demos in HTML, CSS and JavaScript. To test my ideas and see how I could create better demos and get closer to my goal.
But since I didn’t have any good ways of testing the UI, each and every one of the demos ended up breaking without me knowing when it happened. Somewhere along the way I had changed some code, or some dependency had changed, and that was it.
Which meant I needed automated tests for the demos. On the user interface level. First framework I heard about was Selenium 10 years ago on a project I worked on for a customer. It seemed really promising.
But one big problem I remember really well: Mostly the developers worked on fixing Selenium rather than fixing the actual code. It wasn’t stable enough.
Then I started reading about PhantomJS. Based on Webkit (Same as Safari). You could control the browser with JavaScript and by that test the UI with it, but it was lacking a lot of framework for me to actually get things done.
And before I got to test PhantomJS it was outrun by updates to WebKit. PhantomJS had a small group of developers and keeping up with changes in the upstream WebKit was too much to handle. Today the project is abandoned.
Then I found Daydream / Nightmare created by Segmentio. Daydream could record interactions you did in the browser and you could re-play it as automated tests in Nightmare which was an Electron-based browser (which was based on Chromium).
But same problem with Daydream & Nightmare. Too little organisation to keep up with upstream updates. Chromium/Electron pulling ahead, and Segment.io de facto abandoned the project.
It seemed to me that every time I found something interesting, it was already about to die.
But what about Playwright?
Then, a little over two years ago, I found Playwright and started to read about the framework/project. It seemed quite promising, and I wanted to have a go. It's supported by Microsoft — so it’s more likely it will be kept alive and up to date.
The features was just what I needed:
- Recording of UI interactions
- CSS selectors for interacting with the page. xpath if that’s your choise.
- Test-runner of your choise, mine is Ava.js.
- Supports Webkit (Safari), Chromium (Chrome) and Firefox browsers.
- Can emulate smartphones in addition to regular web pages.
- Runs on Linux, OSX and Windows.
I started experimenting with Playwright around 10 months ago and now all but one library with HTML demos have UI regression tests. And I get a breaking test and a notification if something goes bust. Yay!
So, if you're working on a web project and you have some user interface, and you're not doing UI regression testing, I would hope that you at least took a quick look at playwright.dev.
Disclaimer: From what I hear, Selenium is alive and kicking and doing great, so don’t take my word for it if you wonder if it’s a good choice today. But it wasn’t a great choice 10 years ago. And I guess there weren’t many options either =)
