Visual Testing: Optimize Storybook and Win

Hi, I'm Ruben, and I am the lead solutions engineer at Chromatic, and we're the maintainers of Storybook. So, I'm going to start with a story. This is you, and UI testing is hard. There are so many states and variants, responsiveness, loading states, it's all kind of confusing and it all burns down to the ground. Well, hopefully not, but that's how it ends up sometimes.

So, really, there are two problems. Let's say that this is your third week in a row that you've gotten a call and the UI is broken on the website and you need to go fix it, or even worse, the site is completely down. Now, that's not fun for anyone. And then, on the other side, when you're not getting called in on the weekend to fix UI issues, you're dealing with flaky integration tests, and that's not fun either. So, when do you have fun in your job? Doesn't sound like you're having fun at all.

Ideally, we move away from that complicated state and torch it and really make it simple. You just want what your UI looks like and compare it to what it looks like now. And storybook is the first part of solving that problem. Storybook allows you to build your UI in isolation, and it simplifies it because you don't have to worry about back-end logic and API calls, databases, none of that. You're just building your UI. And it works. So, it speeds up the process. And then, chromatic is the other side of that, where it creates automated tests of your UI. Now, what does that look like? I'm gonna show you an example.

So, here we have an application called Mealdrop, and it's an Uber Eats clone. Storybook allows you to create all types of components. You have your atomic components, so, for instance, this button, and storybook has controls which are essentially the props for your component and allows you to play around and prototype things all within the storybook environment in your browser. And you can also create page-level components. So, for instance, like the home page. Or a restaurant detail page. And this has everything in it, right? Your whole page structure. And the best part of it is that it's isolated, and you don't have to worry about inconsistent data or anything like that. Now, that solves the first problem. But it doesn't solve the second problem of how the heck... What do you do to make sure that your UI isn't failing in production? That's where chromatic comes in.

Chromatic takes your statically built storybook, and it compares your UI. And how it does that is you first create a defined state or a baseline of what your UI looks like or what you expect it to look like. And then any time you change your code after that, it will take a new picture or a snapshot and compare it against the old snapshot. And any time there's a change, you get notified within chromatic, and then you no longer have to worry about manually testing, checking different browsers, different viewports, asking your co-worker for their Android phone and checking if it works on that. Yeah, I've done all of that. It's not fun.

So what's really neat here is chromatic can be as specific or as lax as you want it to be. By default, it's really strict. So seemingly, to the eye, nothing changed here. It looks exactly the same, right? If you focus in, before I even go to the section that's highlighted, chromatic knows that there's a slight difference in the image of the hamburger. And if you focus in on that hamburger, there are slight pixel changes. And so it will pick up that minor visual change that you wouldn't notice with your eyes. I mean, you might. My eyes aren't that good. So I would miss that.

Now this brings me to my second point. How do we solve for that in a... How do you take dynamic UI and take a static snapshot and make it work consistently so you're not having to fix all these tests every single time? And that's where Storybook comes in with chromatic. So you have these pages. If it loads. There we go. So the reason that that picture changes is because you have... So we're loading this picture from Unsplash, which is a network resource. And Unsplash can change that image at any time for any reason. The URL is exactly the same. So for all we know, nothing changed with this image. But really it did. And it could be more drastic than this. But luckily in this case it's not. It's just some minor pixels.

And you might just approve it and move on. Well, that's not always the case. And so the best approach to this... There are actually several approaches. If you have the option to load images statically, then what we recommend is loading your images within Storybook as a static asset. That way your snapshots build by build are always consistent. Because you're always getting the same data and always getting the same assets.

Now sometimes you don't have that choice. You have maybe a source set from a CDN. And that's so you can get responsiveness and get different images loaded. And in those cases, we have to get a little bit more tactful about how we do that. So there are a couple options. You can completely mock the data that's coming in and pass mock data to your story. Or you can do a combination where you mock the network request with something like mock service worker. Which we learned about earlier today. And if you do that, you override the network request. So you're still getting a request. But you're saying, hey, when this request comes in, deliver this asset instead. And then the third option is to use the play functions built into Storybook to wait... You can create a utility function to wait for all of the images to load. And then put that in the play function. So you're telling the test to wait until your UI is in a state that you expect it to be.

Now, speaking of interaction tests... What's really neat about those is that you can simulate a user workflow in Storybook itself. So I'm gonna go to these templates. And oh, sorry. The user flows. And like... To get the success page to work in a non-simulated environment, you would have to take so many steps in order to get it working. But here I've defined 22 interaction steps with assertions and expect statements that have to pass in order for this test to pass in CI.

So it covers your entire use case of going from front to back. And I'll even slow it down so you can see what happens. So if I rerun this interaction test, it starts from the very top and goes through the entire user workflow to make sure that this renders correctly each time. And you're testing that ending UI state that you expect your user to see when they run through this process.

Now, the last thing I want to touch on is that interaction tests aren't only for user workflows. You can also write interaction tests for more complex components. So in this case, I had a nav drop down that I was working on. And I was using Daisy UI and remix. And I could not get the modal to close. And I kept trying on my application and restarting and starting it again. And nothing was working. I was, like, oh, yeah, I work at Storybook. Why don't I have a Storybook for this? So I spun up a Storybook, wrote an interaction test until I got it passing and figured out the element that I needed to interact with in order to get it to close. And I fixed that. And so now I have not only a test case, a living test case that will test that my modal opens, but I can extend that in my close menu test. Which takes all of the code I've already written, adds a couple more steps to close the modal. And now I've got a test for both cases.

And that's it.