Test your UI in the REAL Browser

Hi, I'm Gert Hengeveld, a full stack software engineer at Chromatic. Chromatic was founded by Storybook maintainers and provides a platform for visual regression testing and review on top of it. I work from my home in The Netherlands, and because Storybook is open source, a lot of my work is out there in the open. Of course, you can follow me on Twitter for the occasional updates on UI development and best practices.

Today I'm excited to show you something I've been working on for the past couple of months. But first, let's cover the basics. These days, user interfaces are built using components. All modern frameworks provide a way to do component-driven development. Even server frameworks like Ruby on Rails now provide a way to build and use components. This makes sense because component-driven development brings a lot of benefits. You can work more efficiently because not every piece of UI has to be custom-made. You can speed up development by parallelizing work across people and teams. And user interfaces become more durable as you put more thought into component APIs by working on them in isolation.

Despite efforts by framework authors to lower the barrier to creating great user interfaces, testing them is still hard to do right. Testing against a running app can be slow and flaky, and reproducing an error is hard. Complex business logic means there are tons of scenarios and states to consider which you can't possibly test manually. Especially when your UI is in constant development. Existing test tools also leave a lot to be desired. If you think about it, it's silly how the status quo is to run unit tests with JS DOM, which is a totally different environment than the one your user interface will end up running in. Or that you have to install a separate browser to run and debug your interaction tests.

In case you're not familiar with Storybook yet, it's a tool to build UI components. It provides a clean room environment for your components, so you can focus on the way they work, and look in all of their states and variants. It works with all major frontend frameworks. Storybook is primarily built to run on your local machine, alongside your project code, as you develop or test applications. But you can also upload a static version of it to share with colleagues or customers. Storybook can be extended with hundreds of add-ons, to integrate with design tools like Figma or hook up components to a data source, render components like a colorblind person would see them, or generate component documentation. It's the single source of truth for UI components. When you work in Storybook, you'll be building components in isolation. Storybook provides a canvas on which it will render your components. In a typical workflow, you'll run your Storybook alongside your code editor.

Storybook is all about cataloging component variants, and categorizing components to form an organized, searchable component library. This will help everyone find the components they need, making it easier to reuse what's already there.

Storybook is an open source project powered by a large community of contributors. And it's trusted by thousands of development teams around the world. It's used at companies like Shopify, GitHub, IBM, and the BBC. It's even popping up on people's resumes more and more!

Storybook's main innovation is the concept of Stories. Most UI components have more than one variation. A Story is a piece of code that renders a component in one such variation. Think of things like loading states, error states, and functional states, such as a toggle. But there's also edge cases, like extreme values, overflowing content, or missing data. And finally, there's context-dependent variations, such as whether a user is signed in or not, their language, whether they prefer a dark theme, or what their assigned variant is in an A-B test. And to make matters worse, there's also countless possible combinations, certainly not something you'd want to test manually.

Stories are defined using component story format, or CSF for short. It's an open specification supported not just by Storybook, but by many other tools. This means you can write stories once and reuse them in various places. There's no API lock-in, because you're simply writing vanilla JavaScript. You can totally use stories with Playwright or Cypress if you want.

Now let's dive into some code. Here's what a simple Stories file for a Badge component looks like. At the top, we import the component and define a metadata object. This is the default export. This object can be used to define things like the type of data this component needs, where it goes in the component hierarchy, specify default arguments, and wrap the component with a data provider. In this case, we only need to tell Storybook which component we're rendering. And we're also specifying a default label. The badge component has a small and a large variant, so we define two stories, small and large, which only differ in their size property. To write a story, really all it takes is a plain JavaScript object. Now that you know how to define stories, we can add interactions. Sometimes a component has variants which cannot be expressed by passing arguments alone. Maybe you can't or don't want to change the API of the component, or the component is nested inside of another component, such as a page. Either way, passing args may not be an option.

A lot of people struggled with this. And that's why Storybook 6.4 introduces interactive stories. In Storybook 6.4 you can define a play function on your stories which runs immediately after your component renders. This allows you to create stories for complex components and pages. For example, you can simulate user behavior using the highly popular testing library. These interactions run in the browser, so you're actually able to see and inspect the result.

There's even an ESLint plugin to help you write play functions without stepping into common pitfalls. Beyond running interactive stories, we're also working on interaction testing. In the coming months, we'll release tools to make assertions in your play functions. The Interactions add-on will enable you to interactively debug your tests right there in your browser. The great thing about using Testing Library with Storybook is that it runs your regular browser, which means you can use the dev tools you know and love, and you won't need to install any extra software. Using the Node.js Test Runner, you'll be able to run the entire Storybook as a test suite, and when something fails, you'll get a URL with which to reproduce the error.

Now let's take a look at how this works in practice with some live coding. Alright, so I've set up a simple component for an account form with a stories file, and what I want to do now is write some stories for the field state. So let's go ahead and do that. Export const field, and it's just a plain object. It has a play function on it, which is always an asynchronous function. So, what we want to do here is write some interactions against the running component in Storybook. In order to do that, we first need to get a hold of the canvas element, so that we can target our selectors against that container. So that's passed on the story context that the play function receives, canvas element. And we can then use a utility from testing library to get a scoped selector. Import within from storybook slash testing library. And now const canvas equals within that canvas element. Right, so let's save our file and see if we get a new story. So in storybook, new story has appeared, but it doesn't do anything special yet. Alright, we haven't done anything. So let's continue to write some interactions. To do that, we need user event from testing library. And we say await user event.type, because we wanna type into this input field and, well, now we need to provide an element. So how do we get a whole new element? Well, to do that, we need to create a new client.

Well, now we need to provide an element. So how do we get ahold of that input box? Well, what's great is you can just use your browser's regular dev tools here to inspect element. And we can see that this input has a data test ID, which I can just copy and paste. Get me that thing. And then say canvas get by test ID, not get all, just get that one, my test ID, email, and of course, provide a value. Hello. Save my file. And boom, Storybook has instantly re-rendered my component, filled in the form, and also listed that thing in the interactions panel.

Let's do the same thing for the password. Password, hello world, of course. Save my file. Boom. We have two interactions here. We can actually hover over these things, click them, to see what's going on at that particular step. You can step back and forth between interactions. So, let's see what happens when I click that create account button. Boom, it errors, of course. I didn't actually fill in a valid email address and the password isn't long enough.

So, let's actually write a story for this particular state of my component. Because this is a totally valid state that people can encounter and we want to write a story for it. So, export const invalid. This is a new story. I'm going to take a Play function. And again, an Async function. And instead of repeating myself here, I'm just going to use the previous story Play function. So, I need to await that. Build.Play. And the Play function needs a story context. So, that context I will pass along. Save my file, and I should have a new story.

Click it. Boom. Same thing. Nothing's different. Right? Same story.

Now, let's extend it. Again, I need a canvas. CanvasElement. And now I can do await. UserEvent.click. Because I want to click that button to trigger the enveloped state. Canvas. Again, by role. Role. Button. Save my file, see what happens. Boom, it errored. Well, it found multiple buttons on the page because of course there's also a reset button here. So, we need to specify exactly which one we want. Save. And boom, it works. So, now we have a story for this enveloped state.

Let's take a closer look here. There's this error message and it has this tooltip. Let's extend it, extend this particular story to also show the tooltip on Hover. We can totally do that using userEvent. userEvent.hover And now, I don't know again which component, which story, which element I need. So, inspectElement and there's a test ID on it. Copy it, and do canvas.getByTestID Pass it in, save my file and it errored. Well, what's going on here? So, this is a particular issue with writing tests for interactive elements.

Especially if they have an asynchronous process in there. In this case, asynchronous form validation. So, after clicking the button, it takes a very short while for that error message to appear. So, I need to use a different query here. And, luckily, Testing Library provides a different selector to do that. FindByTestID, which is an asynchronous thing, so we need to await it. Save the file, and now it works! So, you'll see the story is updated, and it has actually opened that tooltip, which means the hover worked.

Alright, now let's go ahead and actually write a story for the submitted state. So, what I'm going to do is Export Consubmitted, New Story, and I'm just going to copy the play function from the first one, and correct my mistakes. Great, now I'm also going to submit, click the button, save it, I have a new story. And boom, as soon as I open it, it's submitted the form, and it's done. What we actually want to do now is verify that it worked, that don't just end at the button click, verify that it worked using a test. And to do that we are going to use expect from Jest. So I'm going to import that, import expect from storybook.jest. Again, we need the storybook wrapper for Jest here because that is an instrumented version that will work with the interactions add-on. So what we can assert here is the fact that an action was triggered because what happens when the button gets clicked, this unsubmit handler gets invoked. And that's something that we can assert on using expect. So, await, expect, and then we can assert on the args. And args are passed along to the play function. So I'm going to say args.unsubmit to have been called. Save my file. Let's see what happens. It hasn't been called. Again, this is because there are some asynchronous validation in place, so before the unsubmit handler is called, it takes a little while. In this case, we can fix that using the waitfor utility in the testing library. Waitfor, which takes a function, a callback function, actually. And now I save my file, and now it works.

Alright, let's recap what we've seen. A play function is just a property on your storybook, and it allows you to run interactions and make assertions against your component. In order to use the interactions addon, you'll have to use the storybook wrappers for jest and testing library.

This code really doesn't deviate much from the way you may already be using testing library or a similar tool. We've designed it to stand on the shoulders of giants while bringing more power and better user experience.

By now, you may be wondering how we pull this off. It may feel like magic, but really we're doing runtime interfection and interception. Since we build on top of testing library and jest, these libraries are instrumented to work with the interactions add-on. At runtime, method invocations are tracked and sent to the add-on panel to show a log of interactions. Then, when you're stepping back and forth with the debugger, rather than invoking the library method, it returns a promise which won't resolve until you hit Next. Finally, to prematurely break out of the play function, we throw exceptions which are silently ignored by Storybook. That may sound like a bad thing, but it is the trick to make it all work.

So, how does this integrate with CI? For that, we'll be releasing a Node.js test runner that runs all play functions in your Storybook using a headless browser. And when something breaks, you can review failures in your regular browser or send a report to other tools. In a way, stories have always been like tests in Storybook. With the Interactions add-on, we're only taking it to the next level and we're just getting started. Thanks for listening, and please follow me on Twitter if you'd like to get updates on this. If you have any questions, please come find me in my Q&A session. Thank you so much for that, Goert. It's a talk that I am definitely going to have to watch back because I didn't get a chance to digest enough of it as I would like. And Storybook is a thing that just keeps coming at me left, right and center and I don't get a chance to look at it enough.

Goert, I'm going to welcome you on to the stage and let's have a look at the results from your poll. Fascinating question. End-to-end test, 64%. What do you think about that? First, thanks for having me. I'm really excited to do this. To be honest, I'm not super surprised with the poll results because, let's be honest, TestJS Summit is of course quite a particular audience. So I suspect a lot of these folks are actually using Cypress to do their testing because, frankly, that is the big player, right? That's super popular and I totally get that. So it's not really surprising that we obviously drew a lot of inspiration from what they've been doing. And of course, we have a ton of respect for what they've done. And we are trying to bring part of that into the Storybook universe. And of course Storybook universe has – you mentioned it, right? It's really growing very fast and super popular. Used by thousands of companies around the globe and all very happy about it.

So we are trying to add something into this mix here. That's why I wanted to ask this poll question is what's the way to test the user interface? How do people do it right now? The reason I'm interested in that is of course we are trying to offer a new and additional way to do that. Which is kind of on the edge between end-to-end testing and unit testing. And of course we also would love to eliminate that manual QA for a lot of folks. Because I also imagine that a lot of people are kind of on the brink of they would love to start doing automated testing. But are still stuck with manual QA. Because certain things are hard to test or it's a big investment and such. I think certain things you can only test humanly or manually. But I think the fascinating thing I found here is you weren't very specific with your question right. So all the answers are correct. But there's certain different types of risks. So functionally Selenium and Cypress might be helping you. But obviously it's not what the human sees. So therefore visual regression, manual QA they're going to help. So I thought it was a great question. A great poll and awesome results.

But let's actually have some questions from the audience for you. Because it always puts the speaker on the spot. This is where we find out what does Gert know. So we've got a question from Refactor Eric. What browsers are supported with Storybook interactive stories and interaction testing? Great question. I think pretty much all of them as long as they run Storybook. Which may or may not include Internet Explorer. I haven't tested it to be honest. I suspect it won't really work. Storybook officially still supports Internet Explorer in the particular configuration. But we are moving away from that of course. But we of course are supporting pretty much all the other evergreen browsers. And that's there to stay. Because it just runs in your browser and we intend to support all the major vendors.

Perfect. Do you have stats? Does anyone still go to Internet Explorer for your websites? We don't but we know of other people that do. I work for Chromatic and we offer Internet Explorer as one of the browser options for our customers to use for testing. And still a lot of customers actually do that. So there is still value there. That's the right value. That's the right word for certain people there's still value in IE. It's still there. It still exists. But hopefully, it's on the – Yeah. Maybe we should just stop talking about it. Right? I feel bad for even mentioning it, right? Sure.

So another question by Jane Monroe. Storybook is great for individual components. And are there any add-ons to test the components when you combine them? So basically trying to test like a page as a whole. Something like a whole Figma design.

So sure. Yeah. So there is a ton of add-ons in the ecosystem. And one thing that we are really focusing on right now is making it easier and better to build components – or stories, actually – that compose entire pages and screens or wizards and whatnot. Entire apps, actually.

Storybook is mostly known for building bespoke UI components that you would find in, like, a component library or a design system. So that's what it's well known for. But the really big value is also when you start to use it for app development, where you build your entire web app in Storybook. In fact, Chromatic itself is a very complicated web app, a very complex system. It is completely built in Storybook.

And to refer back to our poll question, at Chromatic, we rely on visual regression testing for 90% of our testing, at least for the front end of the system. Backend, we have unit testing, etc. But front end is pretty much visual regression testing, and that covers pretty much everything that we need. Of course, we have some end-to-end tests, just to verify, like, as a smoke test, to make sure that the thing is not down, right? That it even works at all. But 90% is visual regression testing in our situation, of course, like dogfooding, of course. Well, like I said about the smoke testing, just making sure all the dots are connected.

To give a concrete answer on the question, which add-ons? Mock Service Worker is the one that comes to mind, because it allows you to mock HTTP requests. So if you build an entire page, that page is undoubtedly going to do some HTTP requests to fetch some data to feed it, right? So Mock Service Worker allows you to intercept that request. You don't have to change the code of your app to make this happen, because it lives in your browser and it intercepts the actual HTTP requests that your app is doing and you return mock data so that it is always stable, super fast, no flake, and that's the way that you would build a story for a complex page in Storybook.

Fantastic, so Refactor Eric you had a question about that. Hopefully, let us know if that answers your question. I won't ask it, because I think it does. Julia Bond's got a question. What tests should it replace? They reckon only unit testing, they think. What do you think, Gert? Well, I would actually say, depending on the way that you do your unit tests, maybe. But I would say it is a combination. It's really on the border between your unit tests and your end-to-end tests. So a lot of stuff that you would maybe write a Cypress test for, to test certain flow in your UI, you can write interaction tests in Storybook for that. And unit tests, of course, as well.

And unit tests, of course, as well. So a lot of people write their unit tests in Jest with Testing Library, for example, to test a specific UI component, like the internal state of a UI component. That is, of course, also something that you could or would do in Storybook. So as you develop the UI component, you write a story for it, you were hopefully already doing that. The only thing now is you're adding this play function to add some assertions, the assertions that otherwise would go into a separate test file. Now you just no longer need to write that test file. You put them directly in your story file and you get a lot of those benefits, the same benefits really, at less cost even now.