Testing Our Way to a Better Developer Experience

♪♪ Hey, I'm Will Klein, and I want to talk to you about testing. In particular, I want to share how testing can help us have a better developer experience. I started a company called ToolSpace, a consultancy focused on building great developer tools and improving the developer experience for everyone because it's what I enjoy doing the most.

And working with tests, good tests in fact, is one of my favorite things to have in my arsenal, and it helps me enjoy writing code even more than I already do. I'd like to share with you today how I do that so you can as well. Let me start off with sharing a quick story.

I have a client project where I shipped my first feature recently, and it was kind of challenging to implement, partly because I was new to the code base, but also because I felt like the developer experience was a bit lacking. Some tools needed to be set up, and the tests in particular, well, there were unit tests for some things, but there were no integration tests, and there were no end-to-end tests. So if I made a change, I didn't really have the confidence I'm accustomed to knowing that everything still works. So I worked a little bit more slowly than usual, and I was very careful with understanding the impact of my changes.

And thinking back, I really should have insisted, we really should have some tests in place. I've been there where I've just forged ahead, and, you know, yeah, wish I had those tests. And indeed, after I shipped this feature a few weeks ago, the week after the client reached out to me and said, hey, that work you did on this thing, it broke something. That's very important, and we need you to fix it. And I was like, oh, darn. I was working on something else, I had to stop what I was doing, I had to context switch back to what I had been working on, and I started looking into it. And as I looked into it, I just kept thinking, hmm, I really don't think it was my change, but I didn't have tests. I didn't have tests to know that, to give me that confidence to know everything was still working after I made the change. So I did my investigation, and after a pretty short period of time, I figured out it's not me. It wasn't my change. And honestly, I wish it was me, because if it was, I knew what I changed. I could have figured it out pretty quickly. But instead, I had to tell my team that didn't know what broke it, was not me, which meant that two other developers then had to stop what they were doing and then look at what went wrong. So kind of a classic case of not having the tests that you should have, and your developer experience completely breaking down, your productivity going out the window, having to context switch, having to stop what you're doing, having to figure out what this is instead of building features and doing cool things. This is not where anyone wants to be.

It's one thing to have a test failing, and maybe your test doesn't make sense, and you don't understand it, but at least you know that something's wrong, right? It's much worse when you don't have any test failures, because you don't have all the right tests, and you don't understand what's wrong at all. We don't know when the problem was introduced in this case, and instead, it would have been great if we had some tests to catch it.

This brings me to my first lesson. We could write the test now. Now that we know that there's an issue, we could write a test for this functionality that was broken, but when should we have written those tests? When was the ideal time to have written them and then have them going forward? Let's talk about TDD. TDD is where we write our tests first. And this can be cool, because if we write our tests first, there's this notion in TDD where it kind of designs the architecture of our application and how we're going to implement things. And TDD is great when we know two things. We know what we need to write, and we know where those things we need to write should go. To know those two things, it helps to have well-established patterns and frameworks that rarely change, a set way of doing things that we're accustomed to.

That's not front-end in 2024. I think we have the most competitive landscape yet for frameworks and libraries. And that's a good thing. That's pretty cool, actually. I've worked on a few different projects in the last year, and all of them have had maybe like two or three different frameworks and then a handful of different libraries. Things are implemented in different ways, on different projects, all the time. If you go from one company to another, you could be looking at two very different architectures.

So, yeah, knowing where to put everything at the outset and using tests to guide that, see, that's the thing. You need to know where things are going to go to know where you're going to write your tests. And that's just not great. TDD, front-end, I don't think they mix. I've always had trouble with it. So let's talk about what the alternative is. Testing last isn't great either. Who has heard someone say this? The feature work is done. I just need to write the tests. I have said this at stand-up. I've heard others say this at stand-up. This isn't great, either. Waiting till the end is just making tests into a chore.

And you're inevitably not going to test all the things because you're thinking about finishing the work and not what's the things that need testing. So I have an alternative to suggest. And this is sort of a hybrid of the two. More or less pseudo-TDD, I like to write tests as soon as I know what I'm doing. So what I'll do is I'll start feature development or I'll start to fix a bug. And I'll continue until I have a very clear idea of what changes I need to make to get things working.

And usually, once I've sort of gotten that feature somewhat working, maybe there's still a bunch of requirements to implement, I usually have a pretty good sense, oh, here are the things involved. Here are the parts of the system that are going to have changes impacting this. And now I can get a sense of, oh, here are some of the tests I need to write. And so I'll write as many tests as I can at this point. This part where, this moment where I've sort of figured out what I'm doing with my feature or where the bug lies, this is the critical moment in my development workflow where I've sort of figured things out, but I still have a bunch of work left to do. And wouldn't it be great if I have my tests to help guide my work from here on out? So as I implement my code, I'm not having to go manually test it. I'm seeing my tests go from red to green. I'm seeing my tests pass in real time as I write code. This is awesome.

This is awesome. Or maybe I see some tests fail and I've got those tests written. I've definitely had features where I implement something. This thing I focused on is working. I move on to the next thing. I implement some more requirements. And I go back to that old thing that I worked on a few days ago and it doesn't work anymore. What broke it? I don't know. So I really like using tests as a feedback loop to keep me on track, to tell me when I'm doing things right, when I'm doing things wrong. And definitely in the case of UI, where we're potentially filling out a bunch of fields, clicking some buttons and testing a whole user workflow, doing that manually over and over and over again gets old after just two or three times. And oftentimes I'm making hundreds of changes, sometimes testing them dozens of times. It's great when I have my tests, running those automated tests for me and telling me that everything's good. And of course at the end, I'll check my UI out, but I'll do it once at the end and everything will more or less, it'll always be working if my tests were catching things.

I'd like to compare code coverage to measuring developer productivity by lines of code written, which I think we all agree is not a great way to measure things. Because lines of code that you write does not equate to the quality of code you write. And thinking similarly, high code coverage does not equate to high quality of tests. I still think that code coverage is nice to have. It's useful information to know how much you have tested different parts of your system. Kind of like looking at a PR, seeing what files were changed. That's useful as that.

That's useful as that. But in testing, it often incentivizes the wrong thing, which is writing a test, but not necessarily a good test. And I think it distracts us from where we really should be focusing. Remember our users? This is really about them. Well, I want to make it not just about them, I want to make it more about us.

So let's think about them again for a minute. For building UI, it's usually something to help somebody complete a task. They use this app as a tool. Helps them do their job so they can get paid. Our tool better make sure it's always doing that, and that never fails. And if we're going to test something, we want to focus on what that impact is going to be to the user. If this thing doesn't work, does it matter to them?

So instead of code coverage, we can test it in terms of user experience coverage. Prioritize what matters most to our users and test that really well. What is that one workflow that our product promises that it helps someone do? We better make sure that that workflow always works no matter what. But then maybe there are other things. Like let's say, uploading an avatar and being able to crop it like you can in Slack and resize it or adjust it. That's nice to have. And I hope that continues to work. But at the end of the day, that's nice to have. And if testing that, which testing that really isn't that easy, if that's going to be expensive, well, we have to justify that expense. And if it's not of high impact to the user and the value of our product, well, maybe that shouldn't get too much attention. And sure, test it as you can. But at some point, I would say focus your testing effort on things that really matter and that are actually impactful to the user.

Okay. So we talked about what to focus on with writing our tests and testing the right things. We also discussed earlier when to write our tests and how we should use them to create a useful feedback loop. Let's talk a little bit more about that feedback loop.

Here's a couple more ways that we can increase our developer experience with testing. I like to use my tests as documentation. Now, thinking back to our code, I hope that my code is self-documenting. I hope that things are named well. And as I read the code, I can kind of tell what it's doing. It's telling me a story of what's going on here. And I'm not just looking at things that don't make sense.

Well, I think about my tests the same way, but a little bit differently. So let me look at this first example. The test says that the code returns a string that is the sum of all the row values. This is the most common mistake that I make when writing my tests because I just wrote the code, and then I go say that the test should do the thing the code does in the terms of the code. And this is a major missed opportunity to actually tell the story of what the code is actually for.

Looking at the second example, it shows the shopping cart total. This description is very different, but it's about the same thing at the end of the day. But it's telling me in terms of the user. If this breaks, I know what's not going to work in the UI. I know how it's going to impact the user. This is the way I like to use my tests. And I think of this idea of taking all my test descriptions and having them report out essentially a document that describes all the things that my product does for the user. That's what I want it to do. I don't want it to be a technical specification. I want it to be a user specification.

The test utilities and documentation for testing library are so good, I have to call them out. Testing library was originally written by Kensey Dodds, and it's got different kind of sub-libraries and packages specific to all these other different frameworks and libraries. It's very ubiquitous, and I want to say it does something that a lot of other libraries do very well, which is it sprinkles in best practices. And it tries to guide you in the right direction of testing things properly.

One of the ways it does this is it has a guiding principle. And I'll read it here. The more your tests resemble the way your software is used, the more confidence they can give you. The idea here is to test things as closely as possible to the way your users would interact with the application. This is valuable for two very significant reasons.

Let's look at an example where we don't quite do this. Here we have some HTML output where you can see that there's a data-testid attribute on here. And this is so that our tests can look up things by this identifier and then do an assertion on it. This is pretty common, and I've in the past recommended this. And there are cases where if you cannot find it in a... if you cannot find something in a consistent way, using a testid can be a good fallback, but it should never be the default or the first thing you reach for. And here's why.

So this is great. Our tests will be more valuable to us. They'll catch more problems than we otherwise would have. And they're also going to look at things from the perspective of the user. The user doesn't look for a testid. Can't even see it. They're going to look for text on the screen. So why not do the same thing with our tests, with our test utilities? And there's other test utilities and accessible selectors like getbyrole. This is really important if we're working with navigation and trying to find things that have a certain purpose in the document, in the DOM. And that's the way that, if we look things up by role properly, it's the same way that screen readers are going to look things up.

They'll catch more problems than we otherwise would have. And they're also going to look at things from the perspective of the user. The user doesn't look for a testid. Can't even see it. They're going to look for text on the screen. So why not do the same thing with our tests, with our test utilities? And there's other test utilities and accessible selectors like getbyrole. This is really important if we're working with navigation and trying to find things that have a certain purpose in the document, in the DOM. And that's the way that, if we look things up by role properly, it's the same way that screen readers are going to look things up. So using accessible selectors is a major tool to be bringing into your skill set.

This is one of my favorite ones. Linters. Linters, like ESLint, can help us test our conventions. It's not just about code conventions, though. There are rules and plugins that can prevent whole classes of bugs and protect the user experience. Most importantly, linters give us feedback in real time without having to write individual tests for each of these different things. As we implement something that doesn't adhere to best practices, like let's say we add an image tag to our JSX, but we don't add any alt text. There's no alt attribute or prop, and it's not there. We can be using a linter plugin that tells us about this right away, make sure that we add that. This form of testing is awesome because it's nearly automatic. Instead of having to write individual assertions like, in this view, this image has alt text, we can just configure this rule or plugin once, and then it's going to catch everything from there on out along that dimension of issues.

Here's another one, custom matchers. I remember when I wrote my first custom matcher and saw my test output improve. I just thought, wow, this is so cool and so easy. I have a very simple example I brought over from VTest just to show you how simple a custom matcher is. Of course, when you use a custom matcher, you're actually using it because there's a somewhat not simple scenario, and you want better output than what it's giving you. Just a quick use case example. Let's say you have a shopping cart, and you've got a bunch of things in that shopping cart, and you want to make sure that a discount, a coupon code is applied to everything equally, and you're seeing the discounts applied. You'd have to loop over everything in the shopping cart and make sure that the discount is applied properly.

Let's say you have a shopping cart, and you've got a bunch of things in that shopping cart, and you want to make sure that a discount, a coupon code is applied to everything equally, and you're seeing the discounts applied. You'd have to loop over everything in the shopping cart and make sure that the discount is applied properly. If you actually have assertions checking these values and you write the test code, done this, the output is helpful knowing that the test failed, but exactly what's broken isn't quite clear.

There's two things in a custom matcher that help this. Particularly the message, but the first thing that helps is we can define the pass condition and say this is what needs to be truthy for this expectation to be good. Of course, if this fails, then the message is going to be displayed. This is where you have complete control to provide useful output to the next developer looking at this, which is probably us, and being able to actually specify that, I don't know, maybe the last item or maybe only the first item had the discount applied. Being able to provide very useful output is really powerful, and that's where custom matchers are really coming to play.

I've talked about a bunch of different things, but I'm really still just scratching the surface of all the different ways that testing can make our life better as developers. I want to say, if you ever write tests or work with tests and you feel like it's a chore or it's not providing value to you, that is a sign, that is like a code smell that something here should change. Maybe you need different tests. Maybe you should be using different utilities. Maybe these tests should be deleted and you should write new ones. Maybe you shouldn't have tests for this. Maybe it should be tested at a different level.