Ashes to Ashes, Spec to Spec: The Rebirth of Modern Testing

We've gathered today, tomorrow, thank you. Sorry. So we're gathered today to mount the departure of Mocha, Jest, and Karma. Who knew about Karma? Nobody. Such a short life. Mocha, we loved your minimalism, but you needed some plumbing to plug other tools.

Then came Karma. Karma. Dear, sweet Karma. You did your best to bring symmetry between tests and production by running tests in the browser, but you kept waiting for the browser to connect back, and you never reached beyond Angular Community. Then came the golden child, the savior, Jest. Who heard of Jest? Yes. Well, you promised us zero config, you prankster. You pushed us into snapshot testing. Love your jokes. Jest, then came the great migration. Do you remember the great migration? ECMAScript modules.

We all jumped in the boat with Vite and other tools, and we left you stranded in the CommonJS desert. So we're here to mount, but also to celebrate Vitest as the most downloaded and used testing framework. And it's not thanks to me. Well, special thanks to the V team, so voice zero team, which is now Cloudflare. Anyway, thanks to everyone involved in Vitest, by the way. So you can give them a round of applause. And that's it. Thank you. Okay. So, did I throw my clicker too? Oh, no. So testing does not have to be dramatic. Even though there's a question in today, do we still need tests? Maybe. Well, we can talk about that.

And can be even the pillar for your AI-assisted development strategy. And it's not my topic today, sadly. Today we're just focusing on Vitest and some other tools.

Anyway, so my name is Yunus. I use Internet and social media, so I have accounts there and I post things there, so you can follow me. I have a YouTube channel where I share some of my experiences and stuff and recipes. And I live in Lyon in France. Any people from France here? Yeah. Always together. All day, all day. Okay, good.

And I'm a software cook. And I love to describe our job as software cooks because we're always like trying to find the right ingredients. And people are like, oh, you should really try this. You should put this spice everywhere. Like, no, maybe that's too much. So a little bit of AI here, maybe too much. A little bit of that testing framework, maybe too much, et cetera.

So you have to find the right balance. And amongst the ingredients that I love most, well, it's weird to say it here, but there's Angular, as you mentioned, and NX. And it turns out I'm a GDE and NX champion, which is more like we're building the link between these tools and the community. And by the way, who's using NX here? Oh, good, good, good. Tell your friends about it because it's really great. And I'll be at the booth there. How do you call that? Like the Q&A box is there. So if you have any questions about NX, I'll be happy to talk about it. It's more of not just a modern repo tool, but it's more of like I call that a build companion, you know, in your kitchen.

And for a living, I give trainings and coaching, and I coach teams to improve like the development workflow and the testing strategy and the architecture, et cetera. And I try to make all these things as less boring as possible because software is not always exciting. And my company is Marmcode. And you'll find my workshops and more about my coaching offers there. But that's another topic.

So today what I want to do is I want to search for recipes and filter for them, and then the app shows the recipes. That's it. And I want to test this thing. And I'm running my test with Jest. Who's using Jest today? And who's using VTest? Okay, good. So let's cook. I'll just need my apron. You know, coding always gets dirty. You never know. It's going to work. Okay, I need an ID here. Here we go. So here I have like a workspace. I have my app here, and I have my tests that are written there. And I will just try and run them. And I'm using, it's in mode.

Okay. And I'm using PMPM test that will run NX tests, that will run VTest, doesn't matter. And I will run my tests like this. Uh-oh. The test is not passing. Oh, no, I didn't. Oh, okay. So my tests are still using Jest, and I didn't migrate. Oh, God. Did you ask for me? Why don't you use a codemod? It takes a few minutes to migrate most of your tests. Why not? Good idea. But before migrating Jest to VTest, one solution.

Are there people who are using Jest who want to migrate to VTest and just don't have the time or the tokens for that? So, well, about the tokens, it's also another topic because tests for me are like the pillar, the source of truth. So it's something that I want to review more thoroughly than anything else. And I'll be very careful about just something that migrates thousands of tests. So one good step is if you're in this migration situation, is here I have my Jest configuration, and I will just filter the tests that are called Jest.

And I will rename the tests that are broken because they use specific Jest APIs. I'm going to rename this one. I think this one is broken, too. And this one, too. And that's it. And here I can just run Jest for my old Jest tests, and I can run VTest for my new tests. So that's just a little hint. You don't have to go big bang. You don't have to migrate immediately. You can migrate progressively. Now, about the code mode, another approach that was given to us is to just run this real code mode, but it still works. And it will migrate many things from Jest to VTest. Because one lucky thing is that the VTest team made it easy by trying to mimic the Jest API as much as possible so that the migration is straightforward. And now let's run the tests.

Oh, no. Okay. Now it's passing. Sorry for that. And now we're happy with this. And you know why? I have some tests here that were migrated like this one. And one of the changes is that it was using fake timers to mock the time. And it switched from Jest to VTest. So some migrations are easy like this.

But how do you think that this test here, the recipe search spec, survived? It's a test that tests the whole search page, but without calling the backend. How was I able to have the same code that works with Jest and VTest and still be mocking things? Well, it works. Because here, what I'm doing is that I'm always overriding whatever you use, React contacts or whatever you use, like a dependency injection system or even a singleton system, very simple as this one.

I'm just overriding my abstraction layer that makes the API calls or whatever it's doing under the hood. I don't know. I don't care. Is it GraphQL, HTTP? Doesn't matter. I'm replacing it with a different implementation, which is a fake. And I highly recommend this pattern. I don't have time to dive really deep into this. But it doesn't involve anything specific to the framework. It's just like another implementation that mimics the behavior, not just the API, not just the structure.

How many times do you spend debugging mocks where you have like a whole response there in a fixtures folder and it's broken and you have to update it? And another pattern here is that to avoid the boredom of creating data or letting agents generate stupid data like this that is not consistent or doesn't make sense, et cetera, I have I'm using object mothers, which are a pattern, very old pattern, that helps you build whatever kind of objects for your models but in a meaningful way. So the tests are readable and not cluttered and easy to maintain. So that helps a lot. So this is the kind of patterns that help. And now the thing is I have my tests running in VTest, nothing changed. They're passing.

It's okay. And one of the biggest problems is what if I have a bug or something? How do I debug? So usually here I'm using testing library, for example, React testing library. So one approach is to do this. And then you run your test. And this is the output. But I'm not an HTML parser. I don't know if you have, there are some here. But I'm not good at parsing HTML, so it's not very helpful for me. And then you might have noticed the little green squares.

It's actually a tool called Wallaby JS. It's amazing because it works with VTest, with other tools. And it's not free except for open source. But you can debug whatever values you have wherever you want. So, for example, if I'm here and I can just say what's the value of the data here. I'm sorry. What's the value of the data? And you can see it immediately here. And you can browse it. And it's live. So it helps a lot. You can also say, oh, what's in the filter?

Then you can, of course, control which test to focus on so that you don't get the noise from everything. And you can debug it like this and see what's happening. Oh, I didn't filter. I filtered a test that doesn't go through this. It's not the right one. Anyway, well, here you go.

So you have the status. And you can go through the data. And you're doing a really great job. And they're not talking a lot about their tool, but it's really great. Anyway, but the problem is what if it's a UI issue? How do I debug that?

So that's one of the problems. So maybe if I run the test in the browser, that will help. Well, with VTest, one of the amazing features is that you have different projects. You can configure different projects and have different configurations for different files. For example, here I'm using files that are named browser.spec run in the browser.

For example, here I'm using files that are named browser.spec run in the browser. Files that are named y.spec, these are like integration tests that call the backend. They have different extensions. And they have different configurations. Some of them run in the browser. Some of them run in Node.js. Some of them will run with a different timeout.

So now I just have to rename this test to browser.spec.ts. Sorry. If I run it like this, it's not going to show up because it's headless. And now I can see my test running in the browser. I can even run them again. Cool. And I can interact with it and I can debug more, whatever. So that's pretty nice.

Okay. And the phone. Is that yours? No? Sorry. It's fine. Sorry. Yeah. What? I don't have much time left? Okay. It doesn't matter. So what do you want? Nobody is able to add any recipes today? What? The tests are passing. There are like, I don't know, like 100 people or more that can – can you – the tests are passing. You can see them passing, right? Yeah. She doesn't hear you. It's my manager. You see? But nobody bought any recipe today? Okay. I'll take care of this.

So let's debug this. There's – I think they put the bank. It's on Ashes React 3. The bug is here. And they have a repro, so I'm going to try and run the test again. And the test is still passing. I don't know what to do.

So one thing that I've observed is that here, in order to click on the buttons, I'm using testing library user events. Okay? But you might have noticed that there's a user event from VTest browser. Who knows the difference? Just to know if I'm useful. Okay. Good.

So it's the same API. Nothing changed. I'm going to run the test again. And – oh, the test is failing. Classic. Who are you? I am the other one. Listen to me. Just disable the test. Do not listen to these whispers. In the land of VTest, tests have eyes. They see what users see. They touch what users touch. And what is broken cannot hide.

Why does it have to be always cryptic? Okay. So indeed, what's happening here is that VTest is using Playwright under the hood to interact with the UI. So Playwright has actionability checks because it's using Chrome DevTools protocols to control the browser. And therefore, I have a bug because the next button is hidden because they've changed the CSS.

So whenever it tries to click on the add button, it's actually clicking on the picture of the salad. What will help me maybe is this new feature that's not here yet, which is TraceView. It will come in VTest 5 probably because it's in beta right now. TraceView. Going to run the test again. And look at this. I can see the trace of the test here. And I see that when it's clicking, can you see the button?

So it's really catching CSS and layout problems, which are problems that happen. And this is just because in some shared components in my UI library that I used in my whole organization, I have some CSS issue here. Because somebody added this hate to this object, to the card. And now if I run my test again. Oh, wait a second. Hmm. Is it the trace? I don't know. Anyway. So now it's passing. And now the recipe is there. So it's able to click and I can see that it clicks really on the right button. But yeah, no surprise it's in beta, right? So you can see it's clicking there. Oh, interesting. It's in beta. And which brings another topic. I wonder if you could run your tests, the same test, but in Playwright. Okay? So here, this is something I'm building with a friend. It's called test or not. And you just call whatever you want.

Like you render a component like this. And this is just play write. And then I'm done. So my test is my component is really mounted here. And then I can either ask an agent to finish the test. Or I can use the record at cursor and say, okay, I want to click on the add button. And I want to, I don't know, type here or something. And I can make assertions. Oh, sorry. And I can see my tests running here. And the test is running. I didn't make any assertions. And I can make assertions on the DOM.

I can even make assertions on, guess what. I can call whatever I want in the browser. So I can say, I'm going to select the recipes from my RTK store. And here I have the recipes. And I'm checking that the first recipe has a name burger. Or there's a recipe with a name burger. And I'll look. And my test is passing. And you have all the goodness from play write where whenever you select some selector like this, and you're wondering if it's the right way of clicking or not.

And you can try things like, okay, does this work? Oh, no, it's not the right button. Let's pick the first one. Oh, now it's picking the only one. So that's pretty cool. And this is Testronauts. And that's it. So I just have to find my little slide. And, sorry, just wanted to do.

And one last thing. Oh, sorry. No. Panic mode. Anyway. Apron. Okay. So this is how it works. As I said, you have like the whole magic because VTest is actually using just the automation engine from play writes to control the browser. So that's why it's able to really make the assertions. Oh, no. Sorry for that. Okay, good. So you get this is the interaction. Your tests are running in the browser. And whenever you click, it will go through VTest and VTest will control play writes and interact with your browser and have all the assertions. And if you want to try, you can try Testronaut. Not right now because we still have a little issue to fix for React. And it's going to be here pretty soon. So it's all magic. And just give it a try. And that's it. You can find the slides and also a feedback form about what are your challenges with testing and AI-assisted development. And you can win a license for Wallaby if you're curious about it. So just put your e-mail there. You can find the slides. My free cookbook on testing and NX. And my workshop. And there's an early bird, but that's not the right price. Okay.

Now we're good. Thank you very much. Thank you. That was a great talk. Not just a great talk, the props. It's so funny because I made the joke about the undertaker and the music. I genuinely had no idea that he was going to start with that. Can we give him one more round of applause? I really enjoyed that. Thank you. Thank you.

All right. We have a bunch of questions. First of all, there were lots of really, really good things that you showed, so many tools that people want to get their hands on. And we'll just really blast through as many as we can go in the short time that we have. So the first question is, is there an easy way to see test coverage diffs with VTest so you can measure the impact of changes on coverage? So when you're switching. Yeah. So you mean the that's a really good question. I don't for the diff, I'm not sure. Like on your PR, I'm not sure about that. It's because of another thing. I think that we're obsessed about code coverage and we should be more focused on functional coverage, which brings another longer topic. And code coverage can be tricky, especially when you get agents into that and like, oh, please give me 100% coverage. They will make it, but it's not going to be very useful. But I'll check that out and just send me an email or fill the form and ask the question so that I know who you are and I can show you how to do that. Because previously I was using a tool and I can't remember its name, because there are tools for that. But with VTest, I don't know if there's the diff system. Fair enough. And I know that there's so many stuff where the answer is going to be, it depends, or you want to follow up, so remember that afterwards you can go meet him in the Q&A block outside in the little, I don't know, what did you call it? The little brick room. The little box. Even if you are joining online, there will be a Zoom laptop so the speaker can actually join in and answer those questions as well.

So someone's asked, like you showed us with React, with a web page, any thoughts on testing libraries or ways we can use this with React Native? Have you had any experience with this? Yeah, with React Native it's more challenging because you have, if you use, how do you call that? The web simulator. You mean like an emulator? It's not an emulator, but like just using it in the web. Like the web view? Yeah, like you have a layer where you can render in the browser, but it's not as realistic, so you will have to mock many things, so it's not going to be realistic, so you will not have this experience with VTest. There are other tools like Detox, I think, not an expert of React Native, but I think Detox works for that. Fair enough, fair enough. I know that there's a lot of complexity to some of these questions, and remember you can always get more follow-up afterwards.

This is kind of a general one, and it's maybe a bit of a testing philosophy question, which is what are good rules of thumb for speed and scope for unit testing, right? So lots of people have to decide how granular are we going to get, and I love what you said, there's a difference between code coverage and, was it usability coverage or user? Behavior, yeah, whatever. So tell us a little bit about how you go about prioritizing that speed and that scope. Cool, how many hours do I have? Two minutes and 49 seconds. Okay, so I have a, go to my cookbook, even though I think it's on the Angular section, there is an intro on testing it, but it's very generic, it's not specific to Angular, and one of the concepts is narrow versus wide, and I don't use unit versus integration, because the moment you say unit, people come with lots of different definitions. So my definition of a narrow test is a test that runs in less than 100 milliseconds, and about the scope, it's a test that is isolated easily, you don't have to pop a database in Docker or whatever, and the third rule, which is the most important one in terms of scope, is the complexity. Anyone in the team can debug any test that fails. So this means that you can mount a component with three or four types of children and maybe call a few hooks, custom hooks and everything, but if everything has a cyclomatic complexity of one and it's very straightforward and easy, and the test fails, it's going to be easy to debug and it's going to be easy to understand. So that's one of the rules. So you have to aim for the widest narrow test. So what is the largest test you can think of that can run in less than 100 milliseconds and still be easy to debug for anyone in the team, including agents? Because I didn't mention it, but for example, WallabyJS, they have this MCP tool that allows the agents to instantly know what's the state of the test and also ask about the values of variables in runtime, so it can really debug properly. So if you provide this, it will be able to debug easily, but if you give it a huge scope, everybody will struggle, agents and humans. I really like that.

And going back to your presentation, I kind of used to tell other people who give talks that you have these squirrel moments. And what I mean by squirrel, if you think about the movie Up where the dog is talking, it's like, squirrel! And I think about that when you sometimes see a really, really cool tool or really unique thing, and you had a lot of squirrels, but there were so many really cool tools, really cool things that I'm like, oh, I want to get that visualization in my own ID. And one of these is about what plug-in or IDE tool did you use to show the test coverage in the file explorer?

So that was WallabyJS. Oh, you don't have the slides? So it's Wallaby, like the kangaroos, but they're not kangaroos. And it's a plug-in for VSCode. And also on CLI, it also works for WebStorm, but it's free for open source and paying for private stuff, but it's super cool. And the coverage is nothing. It's just the beginning of the whole features it provides. So it's pretty cool.

And we've got five seconds. Quick one, and this is another. What was GitWipe out? Because you used it a few times in your talk. And I was like, is that a command? I don't know. I even Googled it. Actually, it's an alias that I use. There's another alias when you have all the git tools in ZSH. That's called GWIPE, git wipe. And the thing is that it's a long story, actually. It's because I reset a lot. Like one of my rules is that if I have a good idea, I should be able to commit in five minutes. And I should go into baby steps. So any time I get lost, I git wipe out. It's a really reflex thing. So it resets everything. It cleans the repo. It comes back to the head and you're back to the safety. And you try things and you're back to the safety. Awesome. Thank you.