Visual Regression with Puppeteer, Playwright and Cypress

Rate this content
Bookmark

Visual Regression tests components via screenshot matching. I'll show how you do that in three different libraries/frameworks. Additionally, I will use Storybook to extract the components from your SPA choice.

This talk has been presented at TestJS Summit 2021, check out the latest edition of this JavaScript Conference.

FAQ

Visual regression testing involves creating screenshots of an application or its components and matching them against existing reference images to detect changes. It uses real images instead of just string-based snapshots, allowing for a more accurate representation of the UI's appearance.

Visual regression is crucial because the visual appearance of an application creates the first impression on users. Even if the application functionality is strong, poor visual elements can lead to mistrust and dissatisfaction among users, affecting the overall user experience.

Some popular tools for visual regression testing include Puppeteer, Playwright, and Cypress. These tools help in capturing screenshots of applications and comparing them with reference images to detect any visual changes.

Puppeteer is a library that allows controlling a Chromium-based browser to take screenshots. Combined with Jest, it enables automated visual testing by comparing these screenshots against baseline images stored in an ImageSnapshot directory.

Story undisturbed by other UI elements, making it ideal for conducting precise visual tests.

Playwright, developed by Microsoft, is similar to Puppeteer but supports multiple browsers out of the box. It integrates with its own testing library and handles visual regression natively, marking tests without reference images as failures initially.

For visual regression testing using Cypress, you need to install a Cypress plugin for snapshots. This involves updating the support and plugin directories with the necessary imports and adding configurations to the Cypress JSON file.

Rainer Hahnekamp
Rainer Hahnekamp
9 min
19 Nov, 2021

Comments

Sign in or register to post your comment.

Video Summary and Transcription

Hello, I'm Rainer Haneckamp, a trainer and consultant at Angular Architects. In this talk, we'll explore visual regression testing using tools like Puppeteer, Playwright, and Cypress. We'll learn how to use Storybook and Puppeteer with Jest for visual regression testing. We'll also see how Jest and Playwright can be used together for visual regression testing. Finally, we'll discover how to use Cypress for visual regression testing. Thank you for watching!

1. Introduction to Visual Regression Testing

Short description:

Hello, I'm Rainer Haneckamp, a trainer and consultant at Angular Architects. In this talk, we'll explore visual regression, which involves creating screenshots and comparing them to reference images. Visual regression is crucial for all types of applications, as it impacts the first impression. I'll demonstrate using Puppeteer, Playwright, and Cypress.

Hello, and thank you very much for joining my talk. My name is Rainer Haneckamp. I am from Austria, and I am a trainer and consultant at Angular Architects. So this means I spend most of my time with Angular.

This talk is about visual regression. So this means that we will create automatically screenshots from our application, from our components, and we will match them against existing reference images. This is very similar to what we find with just snapshots, but whereas the snapshots are based on strings, visual regression uses real images.

And as I said before, we are doing a lot of workshops, and every time that I'm introducing this topic, there are two different types of developers. The first type says, well, that's great. I always wanted to see how this works. This is exactly what I needed. And it is very most likely that they are working on UI libraries or that they have applications. There a lot of time is invested into a very attractive design.

The majority though says, well, I'm writing business application or applications, and most of my time I'm using form elements. I'm using grids, and I don't create them by myself. I use already an existing framework like Material Bootstrap or whatever. Why do I have to care about visual regression? If you are sharing this opinion, then just think what happens when you're entering a website and you see that the buttons are overloaded, overflowing, or that the text is overflowing or something like that. Would you really trust this website? Would you really be ready to pay for some of their services? Very likely not. Why? Because the first impression was already a very bad one. And as everywhere in life, first, the first impression really counts. And as it is in applications, the visual part is always the first impression. So don't underestimate that, regardless what type of application you have.

And this is how we are going to do it. I will show you how to do visual regression by using three different libraries, frameworks. I will start with just Puppeteer, then Playwright, and then Cypress. Let's start. Okay, so here we are. This is our application, and what we want to test is this holiday card here. The first action is already that we don't create a screenshot of this URL, because it also contains the side menu here on the left side, the header. And if something changes there, then the test would throw an error, although the holiday card still looks nice.

2. Using Storybook and Puppeteer with Jest

Short description:

We use Storybook to extract components and show them in an isolated way. Puppeteer allows remote control of a Chromium-based browser for taking screenshots, combined with Jest. The Jest Puppeteer preset configures most things, and we have two important dependencies: Jest Puppeteer and ImageSnapshot.

And because of that, we are using Storybook. Storybook is a library that allows us to extract components out of our application and to present or show them in an isolated way. And this is something that I have already done. So this is the Storybook instance. I can click through the different variations, and that's what I'm going to use for visual regression.

Let's take a quick look into our code, how this can be done. So Storybook requires a default configuration. In my case, this is Angular. I need to define all the dependencies, and then I just create the so-called stories, which are the different variations. And for that I'm used here, a self-written factory method.

So we'll start with Puppeteer. Puppeteer is a library that allows you to remotely control a Chromium-based browser. This also allows you to make screenshots, and that's what we're going to combine here with Jest. You see here that I have already a Jest configuration that has a Jest Puppeteer preset. So everything is already, or most things are already configured by this preset. And in our package JSON we have these two important dependencies. The one is of course Jest Puppeteer along with Puppeteer itself. And the second is ImageSnapshot which is able to do the matching of the two images. And that's everything that we need to do except to write the test.