Testing Library: Everybody Uses It, But Nobody Understands It

Hello, everybody, and thank you so much for joining me remotely today to understand how Testing Library works under the hood.

Before I introduce myself, I want to start with a short story. Once upon a time, it was three years ago, I was looking to make my first open source contribution. I decided to go with a tool that I use on a daily basis, so I decided Testing Library.

While browsing through the issues, I found an interesting one. The issue was a problem in our CI process where the build failed for the next version of React. At the time, it was React 17. I decided that this is an interesting topic. I decided I'll look into it. I debugged both React Testing Library and React to try to understand what happened there and why is our build failing.

A few days later, I had a pull request. It was reviewed and merged, and that was my first meaningful contribution to Testing Library. I remember that day quite clearly. It was around 8 pm. I was extremely happy. I decided I'd go to bed. I was still high on the adrenaline. Didn't sleep quite much, but after a good night's sleep, I woke up happy and cheerful for making my first meaningful contribution. I made my coffee. I sat on my desk. I decided I'd open GitHub and see what happened during the night.

The first thing I saw that caused my mood to immediately change was this issue. Version 10.4.4 causing timeout issue with use effect plus just use fake timers. Then I scrolled a bit more. I saw this one also. Type error, cannot read property, version of undefined and this one also. Schedule function is calling normal priority as a function rather than a callback. And that's not an illustration. All of these issues were open at the night while I was sleeping. And yeah, what a dive into Open Source.

All of them relate directly to my change. They even contain code snippets from the code that I read. And this is an illustration of me at that specific moment. But instead of giving something to Stephen Hawking, I gave it to Ken C. Dodds. And wow, I was ashamed. Luckily for me, this happened around the time React Testing Library had only 1.8 million weekly downloads. So it was nice. At the moment, we have 8.7 million weekly downloads. That's quite a lot.

In this talk I'm going to try and give you some context about how testing library works so you won't make the same mistakes that I made in my first contribution. And don't forget about this story. We'll get back to it at the end.

So after this short story, let's make it official. Hello, everybody. My name is Mattan Boronkraut. I'm a senior software engineer at Microsoft. And today I'm here wearing my testing library maintainer hat to explain to you how testing library works under the hood. If you want to follow me, this is the site where I write some posts. Basically, I write for the draw. I had analytics and I saw that I have around five visits a week, so I decided I'll drop the analytics. I don't want this data in my life. And this is my Twitter handle or x-handle.

So before we dive in, let's align on the basics. So what is testing library? Testing library is a set of simple and complete testing utilities that encourage good testing practices. And our API aims to answer this guiding principle, the more your test resembles the way your software is used, the more confidence they can give you. And we can all agree that confidence is what we're aiming for when we're writing our test. We want to be confident before we're pushing to production. And today we'll understand how we're doing exactly that how we're creating the confidence an API that looks simple on the on top of the surface, but under the hood is quite complex. A few more statistics, DOM testing library, which is our core package.

It's used among all of our packages, has around 12.7 million weekly downloads. And as we already saw, the React testing library has around 8.7 million weekly downloads. We support more than 10 frameworks using something we call a framework wrapper. So it means that we have framework wrappers for Angular, Vue. Svelte, and many, many more. And there are more that were written by the community.

In order to understand how we've built this popular package, let's take an example using React and the React testing library. This is a fairly simple React component. It's a counter component that counts the number of coffee cups I drank today. And to be honest, I only drank one today, but this is the component. We can see that it has a button to increase the counter. It has a button to decrease the counter. And it also has a slider with a min value of 0 and a max value of 6. And 6 coffee cups a day is quite a lot, so I hope none of you are drinking more than 6 coffee cups a day. So let's look at the JSX for a second to explain how this component is built. It has a use state with an initial value of 0. It has an on-click handler that increments the count or decreases the count. And in our JSX we can see that we have an input of type range, which is our slider, with a min value of 0 and a max value of 6. We have a button with the minus, we have the button with the plus, and we also have an output element that presents the actual count.

This is a simple test that aims to check that whenever we're clicking on the plus the counter increments and the user can see that it was incremented. In the first line what we're doing is we're rendering the component to some environment. And we'll touch that in a second. Right after that what we're doing is we're trying to query the element with the roll off button and the name plus. After we have that button in our hand, we're firing an event called click, because we're trying to simulate a click event. Right after that we're asserting that the content that the user actually sees at the moment is 1 instead of 0. So let's try and understand what each line of this test means by diving into the actual implementation. So the render function. We're saying render, but where do we render to? Whenever we're writing our tests, the default environment for our test runners is usually Node. And when I'm saying test runners, it can either be VTest or it can be just whatever you choose. And unfortunately, Node doesn't have a window object.

Node only has a global object. And as you probably know, global object is not the window object. And our components use — and this is just a sample — CreateElement, QuerySelector, getElementById, OnClick, IntersectionObserver, InnerHeight, Location, LocalStorage, Navigator, History, and many, many more. So this is just a sample. And all of these are not supported within Node, so we need some way to implement them or to use them. And that's a problem.

So our components use window functionalities. It means that we need an object. So let me help you with this. I present to you TestEnvironment.js DOM. I believe most of you have this line in your Just Config file. What this line does is it creates a window object for everyone to use. It means that once we have this line in our Just Config, we'll be able to use window within our testing. But what is js-dom? Js-dom or happy-dom, which are basically competitors, are JavaScript implementation of the DOM. Unfortunately, they don't contain all functionalities. If you ever tried to click on an anchor within a test, you probably saw that the location isn't changing. That's because it's not implemented within js-dom. So there's no navigation, and there's also no layout, meaning that all of the components are sitting one on top of the other, and there's no actual layout within that specific environment.

Another problem, maybe the most serious problem, is that js-dom and happy-dom are slower than browsers. And the reason for that is that both of them are a JavaScript implementation of the DOM, and the browsers, real browsers, are usually written with a lower level language, so they are probably much, much faster. Just for an example, a query selector all run within js-dom takes about four milliseconds to complete, and that's quite a lot when we're talking about tests. It should probably be as fast as possible. So now that we've understood where we're rendering to, let's look at the code for the specific render within the Arc testing library. This is the GOAT logo, and this is the React testing library logo. It means that the code we're seeing now is in fact specific to React testing library, and every framework we have has a specific implementation for these specific functionalities. So what we have here, this is a set of code. It means that I removed some stuff that weren't relevant for this slide because it would only make confuse us. At the first line, what we're doing is we're creating a react route. React route is what you're also probably creating within index.js or index.js because we must have a React route to render React components. Right after we have that React route, we're pushing references to the container and the route so later on we'll be able to clean it up.

And we can see that we're also calling route.render, which is basically the render function. So we're telling React, please render this specific UI. We can see that it's wrapped in Act. Act is a testing utility written by React, and we're not going to cover it in this talk. In our return value, what we have is a container, an unmount function, a rerender function, and the entire query, the entire testing library query is bounded to that specific container, bounded to that specific element that you've just rendered. So this is quite simple. I believe that most of you can probably take this example and create a render function for every framework that you're using if it's not available yet. So this was the render part.

The getByRoll part is much more complex, and let's dive in. This is the DOM testing library logo. It means that this code is not framework specific. As long as we have a DOM, this code can run and it will work. We're trying to query an element with the roll button and the name plus. And if you remember our UI, this is how it looks like. We have a slider, and we have two buttons, and we also have the output element. If you remember, we didn't define anything related to rolls there. This was just buttons and an input. So let's look at accessibility tree for a second. The ByRoll query is the best type of query, since it's built on top of the accessibility trees. It gives us the most context and the most power to create the best tests and the most confident tests. We can see that we have rolls there to find here. We have heading roll, we have slider roll, we have button, status and another button. But we didn't define all of these in our JSX. It means that something, somewhere within the process of us creating the component, these rolls were added. So in order to understand the implementation of ByRoll, we need to first understand the specification of rolls. What does it even mean, a roll? All of these rolls are defined in the World Wide Web Consortium, which is a group that defines the specifications for the web. And specifically, the roll area is a part of the area specification, which is accessible each internet applications. A roll adds semantic information for assistive technologies, and it can either be explicitly defined, which we'll see in a second, or it can be implicitly defined. So when we're saying that a roll is explicitly defined, what does it mean? It literally means that we're writing attribute, roll equals alert, meaning I want to define that this dev has a specific roll. So how do we in testing library query elements by their explicit roll? Whenever you're writing screen get by roll button, we're transforming this specific query to a document query selector all with a roll button.

So that's quite simple, right? Nothing fancy here, no magic, this is what we're doing. An implicit role, on the other hand, is much more complex. An implicit role means that the role is defined implicitly. And how can a role even be defined implicitly? We're lucky enough to have semantic HTML and semantic HTML usually gives more information to the user than the actual element itself.

So we can see that we have here an example of coffee types. We have ul and we have also a few lists. In semantic HTML, ul implies list role, and li implies list item role. So this means that we in testing library need to have some sort of mapping between an element type and that specific implicit role for that element. And we do have something like that. So whenever you write get by role button, we transform this query to this. Document query selector role input summary button. All of these element types can imply button role.

So what did we get if we combine the explicit role and implicit role? Whenever you write screen get by role button, under the hood what happens is, we're writing document query selector role. Give me all the elements with the explicit role button, and also give me the element types, input, summary, and button. So you're telling me that get by role is just query selector role? Well, not exactly, and this is where it gets complicated. Let's go back to the JSX for a second and go over the query that we just written, line by line or part by part to understand what we get. This is our JSX, and let's go over the query part by part. At first, we try to query all the elements with the explicit role button, and we can see that we have no elements with an explicit role of button, so we can move on to the next part. Now we're searching for elements of type input. We can see that we have one, so we save that aside. Next, we're looking for elements of type summary. We don't have any. And lastly, we're looking for elements of type button, and we have two of those. It means that as a result of this specific query, what we're getting are three elements.

So we have one button with the minus, we have another button with the plus, and we also have an input of type range. So what do you think? An input of type range is really a button? Well, not quite. Not every input implies button role. This specific input of type range implies slider role.

So what am I saying? This query should probably be better. Document query selector or input type button saying give me all the inputs with the type because they are probably certainly buttons. And maybe you know where I'm going with this one. But let's play a game. What's the implicit role for input of type image? Is that A, an image, B, generic, C, button, or D, no implicit role. Let's take five seconds and see who's right. Okay, so, if you chose image, generic, or no implicit role, I'm sorry, but you're wrong. The answer for this one is a button role. Input of type image is, in fact, button role. It implies button role.

So what I'm trying to say here is that implicit role is a complicated thing. And building a selector to fit all cases will be unmaintainable for us within testing library. So what we're doing after we get the list of all the elements that answer the query, the broad query, we iterate over the elements and we try to filter them by the specific attributes that map to the specific implicit role. And after filtering, this is the list we have. We have two buttons. But that's not yet the element we wanted. We forgot about one specific type of the query. And that part was the accessible name. We're looking for the specific button with the accessible name plus. And it's important to know that an accessible name is not the name attribute. And I'm saying it because we get around one issue a month saying something like, I've added the name attribute, but I can't I still can't query that specific element. That's because an accessible name can either be implied from content, which is called name from content, or it can be implied from ARIA semantics, saying ARIA label, ARIA label by and that's called name from author. Luckily for us for button element, it's implied from the content. So whenever we say name plus, we know in fact that this is that this is implied from the content. And now we have our element. One last thing happens here.

Whenever you try to query an element, we want to verify that it's in fact visible and accessible to the user. So what we're doing is we're iterating, iterating up the tree, because if any of the ancestors of this element was hidden, it means that this element is also hidden. So we don't want to give it to you. So if any of the ancestor or this specific element is hidden, we won't return it to you.

So let's summarize. The first thing we're doing is we're building a query based on explicit and implicit role. Right after that, we're filtering out the elements that have the wrong implicit role. We're calculating the accessible name and the description. And we're checking the tree for accessibility. That's why the Byrol query is the slowest of all queries. And that's also why it's the recommended query. It's the most powerful query as it imitates the user behavior as best as we can within our tool. It's also important to know that we're currently investigating how we can improve the performance bottleneck that GetByRole actually causes.

A few more things. Fire event. Whenever we're trying to fire an event, what happens under the hood is we're actually running native dispatch event, saying all we're doing is whenever you write fireevent.click is element.dispatch.event. This is basically as the spec works. And the Fire event is a way to fire DOM events. It's a low-level API, meaning that in most use cases, you should not use it. You should probably use user event instead.

And the last thing that happens within our test is as long as your test environment supports after each, we're running clean up for you. So if you remember this line from our render part where we pushed references to the container and the root to an array, what we're doing in the clean apart is we're looping over that array, and we're calling root.unmount to unmount the React tree, and we're also calling document body.removeChild to give you a fresh and a clean slate whenever you're opening or whenever you're starting your next test run.