Mock Service Worker 2.0

Hello, everyone. It's a pleasure to be here and thank you for joining. Today, I would like to speak about API mocking, standards, JavaScript and, of course, MockServiceWorker.

First things first. My name is Artem. You can find me as Ketanaito pretty much everywhere, including my humble blog at ketanaito.com. And you may also know me as the author of MockServiceWorker.

Out of curiosity, can you raise your hand if you've heard about MSW? Wow, that's a lot. You will enjoy this talk. Yes, so for those who haven't heard about it, MSW is an API mocking library. So in essence, it allows you to intercept requests and mock responses, basically control your network for whichever it is you want, testing, prototyping, debugging.

And it has a bit of a unique approach to API mocking. It introduces API mocking as a standalone layer. So if you think about how you control your network during testing, for example, well, you have a bunch of these different tools, right? You have tools like Playwright and Cypress and Vitesse and Jest, and they all ship with their own API mocking capabilities, which are amazing. But the thing is, once you achieve this full test coverage of your product, you start to notice that this network descriptions you do, they kind of repeat. In essence, you just want to do one thing, describe the network, but you're using different APIs from different frameworks, and they don't know about each other. They can't communicate. It's very little you can reuse. And that's precisely what MSW solves. It lifts this intention up. So you have a single layer of your network description, and then you integrate it in particular tooling. And as a cherry on top, it achieves API mocking without resorting to this nasty window.fetch patches, and instead it relies on the standard JavaScript API, like ServiceWorker API, if we're speaking about the browser side interception.

But how do you do that? How do you describe the network with MSW? Well, you use these functions called request handlers. I will show you an example. So this is a very simple request handler. You can see that the intention here is to intercept a POST request to the slash user endpoint. And then we have this callback function that should produce the response. We have request, argument, response, and context. So what we're going to do with this request, we will try to read its body as JSON. So MSW will try to be smart here, actually, and see what content type header your request has.

And if it's application slash JSON, it will try to parse it and give you the object back. So we're accessing the name property here. And then to return a mock response, we use this REST composition function to compose a bunch of context utilities, like this context.json to return a JSON response. So this is a very, very basic example of your network description.

And one thing I like to do when using any third-party libraries, I like to do this fun exercise. So I take a very basic example like this one, and I remove everything that's not standard JavaScript. So let's do just that. Oh, not really much left. Basically all that was written there is still valid and functional, but it's all abstractions. If you think about it, REST.POST is an abstraction. It's something library-specific. The same as this call signature of response resolvers. Nowhere else in JavaScript do you use this call signature to handle requests. The response function, as much as I love functional composition, it's not as common in JavaScript, for better or worse. And, of course, the context utilities, a completely made-up API. And this API has been functional and working for years.

In fact, this year, I think, MSW marks its fifth anniversary. And you can say, well, if it's not broken, just don't fix it. Not a big fan of that. In fact, working in open source, I think, has taught me a much more important motive. If it's not educative, don't ship it. Because my goal as an open source maintainer is to share my ideas. But I also want all those developers that like my ideas and use my ideas to get better at JavaScript, at the language, not in my little projects. Because, if you think about it, JavaScript has got a lot of improvements over the years. And you can see this very apparently in the directions that frameworks like Next.js and Remix are taking. They're ditching those custom abstractions. And they're betting more and more on JavaScript because it's amazing. It just so happened that MSW has been used a lot. It has over 90,000 projects on GitHub alone using MSW. And more than 2.5 million downloads on npm every week.

That's an insane amount of developers that have to go through the docs, learn about these custom functions that they never heard before and probably will never use again apart from MSW. That's just a lot. And I just realized recently, we live in the modern era where JavaScript is everywhere around us. And I was booking tickets to this conference and I interacted with JavaScript. I called my mom the evening before and the Messenger app has JavaScript parts in it. So if I happen to have a little influence on people who write that JavaScript, why don't I teach them to be better at it? And that's precisely what I tried to do about a year ago. So I set off on this mission, we need to improve MSW.

As much as amazing it is, it can be better. So the primary perspective there was one of the key points that we had in the library from the very beginning. You should mock your network in the same way that you use it in production, for example. So let's take a look at how you do that.

In the browser, most likely using Fetch API or an abstraction or a Fetch API, it's a fantastic API. Or you may be using XMLHttpRequest, which is still great for certain situations, albeit being a little bit older. And in Node.js, it's a bit more tangled. The main way to make requests in Node.js is the HTTP module, which you're probably not using directly for a good reason. So you rely usually on third-party libraries to do requests, to handle responses. And one particular thing I learned when writing a request interception for Node.js is that there are more ways to make requests in Node than there are stars in the sky. And unfortunately, that's true, especially when you dive into like user land and what people are building their own libraries and they use Node.js in all sorts of ways, sometimes according to the specs, sometimes not. But what we try to do in MSW is to take all these different ways you can define requests and responses and kind of bring them to the common denominator. And we introduced this concept of isomorphic requests and isomorphic response. Basically, again, an abstraction just to handle all these differences. But it turned out to be quite a mess. Because yeah, you can represent very basic requests and responses, but there are a lot of features that developers want when working with network. You want to be able to handle array buffers, files, form data, uploads, streaming, maybe abort the request. And we had to kind of implement it by ourselves. So no matter how you declare requests, you still get those capabilities. It was a huge mess.

And just when I was losing faith in all humanity, something amazing happened. Node.js 18 was released. Now, this was a grand release, especially for me, because I was deep down in refactoring in MSW and we still promised support for Node.js 14 and 16. So there was no common ground really to represent network until this happened. And of course, I'm talking about one of the biggest features in this release. It was the fetch API, this very fetch function that you are all familiar with, I'm pretty sure. But it's not only that function. There's been a lot of other things added, like readable streams, headers, working with abort signal and URL. And if those things were not introduced by this release directly, they were facilitated. They were much easier to use through the fetch API. So this was amazing. I was very happy and I thought, hey, we can use fetch API as the common ground for both browser and Node.js to represent the network. And I began refactoring and employing one of my favorite practices, throwing code away. And it's been fantastic, honestly, just throwing all the subtractions away and all the docs because you don't need them anymore.

And then a year later, we arrived at MSW 2.0. On the surface, it's pretty much the same request handler. This is, by the way, the same request handler, just with a new API. So we still have the intention to intercept the post request to the user end point. But in the callback now, we just get this request object. And you may be wondering, like, what kind of abstraction is that?

So if you take a look how we work with that request object, you can see something familiar. It's just we're calling request.json and we're awaiting that promise. It kind of looks like fetch API because it is. It's just regular request instance. So we read it as JSON and we get a name property. And if we want to define a JSON response, we just use the static JSON method on the response. Again, the standard response API in JavaScript and return a particular object. So let's take this example and do the very same exercise. Throwing away everything that's not JavaScript.

Boom. Sure, the way we declare the contract of how we intercept requests, it will be contrived to some extent. JavaScript, to be frank, doesn't have to deal with that. It's quite specific. But if we take a look at this heart of this request handler, at this mock logic, it's all just standard JavaScript. And that's what matters. Because the more logic you write here, the more regular JavaScript you will be using. And it became apparent very fast that by using the platform, everybody wins. And I'm talking about developers and I'm also talking about maintainers, such as myself. Developers, well, for once, they have a shallow learning curve. You simply don't have to waste your time in the docs reading about all these fancy abstractions library authors created. You can just use JavaScript. You get features for free. Like if you want to mock, I don't know, a readable stream, you can just do that. Because that's what Fetch API allows you to do. There's no need for MSW to implement that explicitly.

And of course, you just get better at the language. As I mentioned before. So, you interact with this primitives more often. You get more familiar. And this knowledge that you take from using MSW, you can apply it even outside when I don't know, writing loaders in Remix or handler requests in Next. Those all rely on the same Fetch API.

And maintainers, of course, we can ship less code. And it's great because it means less code to maintain and less bugs. We get better compatibility. Because if everything in the ecosystem one day relies on the same common ground, this means that people, our users, can just use the same logic, the same utility functions, the same tests, everywhere across the stack because it's the same dependency on JavaScript.

And despite common belief, open source maintainers aren't this wise, all-knowing creatures. We do make mistakes and we do have to iterate and improve a lot. And the more we ourselves interact with the language, the better APIs we can design for everybody to use. If you are excited about this direction, or if you've never tried MSW before, you can do it right now. Just npm install MSW and you can mock any APIs anywhere across your entire stack.

If you want to dive deeper, though, I spent the last year working with Ekhed on this fresh new course that's called Mock REST and GraphQL APIs with a mock service worker, where we're going to build together a movie streaming app completely mock first, so against mock servers. And we're going to dive into the basics of getting started with the library and intercepting your first request, to more advanced concepts like mocking streams and dealing with GraphQL queries and mutations and even schema-first approach in mocking. So it's a fantastic place to get started. I encourage you to watch this course and I'm sure you will learn a lot.

Of course, you can visit the project itself at mswgs.io and on GitHub, too. Read about it and if you believe in the mission we're doing here, support it both financially and contribution-wise, which is really the best support. And yeah. Thank you.

So, we'll get right into it. There's quite a few, like I said. The first one here. So, are there any big features that are currently missing from MSW when compared to older and maybe more mature alternatives? I think there's a lot of expectations features, and that's a pun. Like people are used to, for example, using NOC in tests, and sometimes people want to have this expectation layer. Like you want to perform a request, but also expect that it has a particular payload or it has been performed once. This is a deliberate choice that MSW doesn't ship this and instead I would rather have an ecosystem package that extends the usage, because not everybody needs these expectations. And so, this is something that's missing. And I think one of the most anticipated features that I hope I will have time to work next year would be, of course, the VAPSocket API support.

Okay. Great. That's good to hear. For sure. This next one here. Can we auto-generate MSW code by monitoring network requests from our application and updating them when the response structure changes? This is a really good question. I know a lot of folks are doing like crazy magic, like deriving MSW code from your GraphQL type definitions, from your open API specifications, from your packed contracts. So, you can find all of that online. There's again, ecosystem packages, people building this. As of native support, I think, like letting you on a little teaser, I would much rather have HTTP Archive as the common ground. So, maybe in the future, there would be something around that.

Okay. All right. We'll keep an eye out for that, for sure. No promises. Okay, so this next question. Why not just mock the response instead of mocking the whole request? Hmm. I'm not sure I quite understand the question, but I can elaborate a little bit. So, with MSW, you are not actually mocking requests. This is something I briefly touched upon on one of the slides that we use the ServiceWorker API, if talking about the browser interception, that allows your application to actually make requests. So, there's no stubs, and your requests happen and arrive at the network and just receive response from the ServiceWorker.

So, the only thing you're mocking is the response. The request part is there just to have this contract of request-response collocation. Yeah, okay. That's good to clarify then. Yeah. All right.

Does MSW give support to TypeScript yet? MSW always supported TypeScript, but I think my fault was not providing more materials on that. I wrote one article on Dev.Too, which I guess is one of my most viewed articles, and I should have known better, about using MSW with TypeScript, but it was, like, based on the previous version. So, in the new version, I do want to do something better about it. It would be nice to have code examples in the docs that feature TypeScript as well. So, this is a contribution tip for you, if you want to add it. Yeah. So, it will definitely work better, but MSW is written in TypeScript natively, so it always had this generic support for request body, response body, body parameters, and even GraphQL queries and variables. All right.

So, if you're new here, docs contributions welcome. Always a great place to get started with open source, if you haven't yet. So, yeah. The questions just keep coming in, so we'll keep going through them. Can you provide a use case with MSW and contract testing on CI pipelines? Yeah, absolutely. I think MSW is just a way to facilitate contract testing. So, you may be, like, in itself, it doesn't do that because it's a very simple tool of request response contracts, but you can use some more advanced tools like PACT I-O, and you can find a way to integrate them. I know that the PACT team has an official integration. I believe it's PACT-MSW that you can find on npm. And I believe it utilizes MSW as just an interception algorithm and uses PACT for contract testing and CI. So, you absolutely can do that, especially if you like contract-driven testing. Go ahead.

So, it sounds to me like, really, MSW is designed to do, like, what it does very, very well and work really cleanly with other tools to achieve whatever use case that you need. Yeah, and one of the ways that we try to improve that is exactly betting on the same standard API so everybody gets the compatibility. Yeah, absolutely. As I'm curious, if Node hadn't come out with these changes, do you think that you would have taken a different approach at some point or just kind of kept working on trying to make it as usable and compatible as possible? I think I would spend another two months pulling my hair out, and then I would contribute to Node with FetchAPI.

Okay, there you go. Yeah. You know, if it's not getting done, do it yourself, I guess. But luckily, like you said, it got done, so that's great to see.

Okay, this next question, do you recommend using MSW with unit tests or only with integration? I guess just as with testing levels, it depends where you draw them and what kind of product you're building. Conventionally, you test network code in your integration tests. Your units should really be isolated, and that includes isolation from HTTP interactions. So people do test heavily and use MSW heavily in integration tests. You can also use MSW in end-to-end tests. I'm particularly fond of this concept where you run your end-to-end test suite against mock and then against production. It can help you catch the little deviations. But of course, if what you're building really has to have network code in its units, I suppose of course it's up to you. Yeah, I think we're required to have one it depends answer per Q&A. So that's a... There you go. Because it always does. It always depends, right? Yeah. Yeah.

Okay, so this next question, since you believe we should work based on standards like JavaScript, what's your take on TypeScript? And so we kind of answered this earlier that it's already used within MSW. But can you maybe share your insights on that or thoughts on that more? Yeah, I do love TypeScript, although it's a superset and a lot of people understand it. They view it as a different language. To me it's a crucial thing as a maintainer, because I have all this functionality to maintain and types provide me with static tests, basically. And when you have a lot of features and a lot of people depending on your app, it's really crucial that you keep the thing working. And type contract is one of the things. Apart from contributing to the developer experience, of course, when you publish these types, I know there's a lot of movement to GS doc, like using just type comments. I haven't tried that. I know it works for some teams. Again, do it yourself. Make your own conclusions. There is nothing really in TypeScript to be afraid of, because at least as far as I know, it doesn't introduce any runtime features.

It's a completely built-in language. It strips away the types and every feature you got is a JavaScript feature. So we are betting on JavaScript, even when using TypeScript.

All right, great. Thanks for sharing that. So this is a good one here. What's at the top of your wish list, so maybe if you were to build it yourself or wish for it, for the future of JS API request response fetch features?

Oh. Hmm. I do wish we got upload progress to fetch API. That's probably one thing that makes me use XMLHttpRequest still. And I hope that maybe one day we will. Okay. And hopefully too. If not, maybe we'll see a PR.

All right. And what do you think about Chrome developer tools and mocking?

Yeah. I actually heard that the team at Chrome introduced a lot of cool things recently. You can do mocks straight in the network tab, and then maybe even save them or something. I think it's really cool. But I see this as primarily a debugging instrument because it's faster. It's more accessible. It's DevTools in the end, so it's your tooling to debug and build. But as far as the comprehensive solution, I still believe that API mocking doesn't belong to a particular tool, even if it's a browser tool. It can be a cool feature, but if you are building the whole product, you are testing the whole product and you want this network description to be consistent, just like I demonstrated on the slides. It's not just plugs in different areas. It's just one layer, then reused everywhere else.

Yeah. And it sounds like you want to have that kind of intentionality from the beginning when you're designing that out, right?

Yeah. Yeah. So kind of more proactive versus reactive, so to speak.

Yeah. That makes a lot of sense. But yeah, there's a lot of cool stuff in Chrome DevTools these days. For sure. Yeah.

All right. So will you support mocking WebRTC? I think there is an issue about it for like dating two years ago or something. Maybe it's something else. I think the stance on MSW, whatever support you bring to it, is very simple. It should provide you the primitives and you can build on top of those primitives to achieve whatever you want. So if you want DRPC support, I'm almost certain you can do this with MSW right now. But it doesn't mean that MSW should ship it natively. It's quite an important decision to draw this line where the library responsibility ends and where user land begins with all this crazy requirements and use cases. Yeah.

Like we were talking about earlier, right? It does what it does well. Yeah. It integrates with other tools. And do you think that there's... Instead of maybe trying to build something that does it for itself, maybe just illustrating those use cases, like with documentation or examples?

Absolutely. Like one thing that we can do is just show examples and we actually rebuilt the website over the past year. So it's a completely new website. It has docs. It has search. Oh, that's a big one. Yeah. And I'm trying to add more and more recipes, like streaming and like polling, using generators, a lot of stuff that I know people are using, but not everybody knows about. Like I know that we had a chat with folks from Apollo and we were iterating on how to improve schema-first mocking. And over the chat I realized that I'm not doing a good enough job to promote that you can do schema-first mocking even right now. You can do mock resolvers, mock schemas, mock types. It's just a documentation issue.

Yeah. I feel like there's kind of two types of developers or testers. Some will... If it's not in the docs, they don't think it exists or they're afraid to use it. And then some other people who will just use it for anything and everything, including maybe things you shouldn't use it for. And so, yeah. So I think that there's some people who probably have already figured out that it works together, and there are other people who could use a little bit more guidance.

Yeah. I think as a maintainer, my job is to bridge this gap and bring these people closer. Yeah, absolutely. All right. We still have more questions and have a little bit more time for them. So this next question. So Venkat Subramanian said once, knockout before you mock out. I haven't heard that one. Could we fix the root problem in Fe with better architecture and dependency injection? I think there's nothing particularly wrong with mocking. It's just a technique to establish boundaries for you to tell the test where the test should end, where your system should end. So on its own, it's not evil. I know it historically has this air of being hacky and stabby. And this is something we are trying to solve in MSW by using service workers, by using a module like mutation in OGS. But I guess it's still another one. It depends on what you're testing and how you built it. If you constructed your app to be friendly to dependency injection, then of course you can utilize it during testing. But you need to acknowledge this creates a different testing boundary now.

Yeah. I think it's like you said, with any tool or approach, it's not inherently good or bad in its own, but how you use it. If you have a hammer, it can be used really well for some things. Maybe don't do it for something that requires more precision and that it doesn't make a hammer good or bad. Exactly. And so, yeah, I think, again, we'll give you two this time for this Q&A.

It depends. So, all right, this next question, what is the best tool for API automation according to you? API automation. I wonder if you mean testing or like complete workflow? I know that Postman has been around a lot. I use it in the past a little bit. I don't really know what to recommend. There's a lot of them out there. I think I'm going to say it. It depends, right? It depends on what you're looking to automate, what the use case is that you are trying to automate for your API. If you're doing automated API testing, if you're doing any kind of like migrations, I think it will just depend on- Yeah, especially like with the rise of AI right now, I'm pretty sure there are some startups building really cool, like semi-intelligent features of like scrapping your API and letting you discover them. Making sure that new projects are rising every day. Yeah, absolutely. And since you brought up AI and there wasn't any specific questions about it, how do you see that kind of maybe potentially impacting how we think about day test data or mocking or in any kind of way? Is that something that you would see integrating well or not? I think AI just as mocking is just another tool you can use to supercharge your experience. I use GitHub Copilot or CodePilot when testing. It does help writing repetitive tests when it's kind of easy to predict what I want to test, but for more complex tests, I would rather turn it off. So again, it's a tool you need to know. It's your responsibility when and how to use it and to understand that you sometimes need extra to pay extra attention to what the AI generates here. Yeah. And I think, I know I mentioned this before, but one of the reasons why people go to mocks is because obviously controlling your test data can be very difficult and generating the test data and creating the seeds. So if there's any way to do that from either test code or application code and handling all that for you, that's something I think would be a very good problem to solve because it's a painful, painful problem. It can be. It can be, yeah. All right. I think we have time for one more question. If you have any other questions for Artem, make sure to get them submitted in Slido. So is an alternative integration with Postman possible? I'm not sure. I suppose it implies MSW and Postman. I think so, yeah. I frankly haven't used it in a very long time to know where can we draw this common line. I'm sure it's possible again, because MSW is a simple tool. And then Postman is a much more comprehensive tool.

I hope Postman supports something like network archives. And this is again like where we can draw the common line maybe. So I don't know. Why don't you give it a try?

Yeah. And Postman's here. So if you want to ask them any questions or chat with them, let us know. Well, if you have any other questions or if I didn't ask your question the right way or we didn't address it, please feel free to chat with Artem, speak your Q&A, and hear around the conference.

But yeah, thank you so much for your great presentation and for being here. Yeah. My pleasure.