Everyone Can Easily Write Tests

Everyone can easily write tests. Hey everyone, my name is Debbie O'Brien, I'm a senior technical PM at Microsoft advocating for Playwright and ensuring you are all writing tests. That tests is easy, it's fun and it's something that anyone can do. So, don't be afraid because at the end of this talk we're all gonna be like oh my god I need to write tests like right now. So, bare with me.

Yes, if you want to follow me, debbs underscore O'Brien on Twitter or X or whatever you're calling it these days as well as all the other social media platforms out there, debbie.code is my website. And let's get straight into talking about Playwright because that's why we are here today. So, what is Playwright? For those of you that do not know, I've never heard of Playwright. Where have you been, seriously? Playwright is reliable end-to-end testing for your modern web apps and it works on any browser. So you can be on like a Windows and you can test on WebKit for example, one API, all your tests run in full isolation, they're really really fast. We've got some amazing tooling which we'll go into today and it is multi-language. So, I'm going to show you TypeScript, but you could write in JavaScript of course or also in Python, Java or .NET.

So, why playwright? I mean, all those reasons that I've just shown you just here, but also there's so many more, but I'm just going to concentrate on those and we'll go through them step by step. And there are so many other reasons, but seriously, you should be using Playwright in the story.

So Playwright auto-weighting, what does that mean? So for example, imagine Playwright needs to click on a button. Now Playwright is like a really, really fast user. You know it's really like frustrated fast users. So what happens when maybe the JavaScript isn't ready or the button isn't properly loaded in the DOM and therefore it's not fully clickable? Well, Playwright will wait for that to be ready. So it's auto-weighting for that element, which means you don't have to write any artificial timeouts or anything like that. So that's really cool.

So if you're using the Playwright assertions, they will retry out of the box. What does that mean? Well, imagine you have expect that button to be visible and it's going to wait for that button to actually be visible and it will retry until it is or until the timeout runs out. So out of the box retrying assertions.

Now, what if you want to test iframes and shadow DOM? Well, it just works. You don't have to install anything, no new plugins or anything like that. You can just test the iframe or test the shadow DOM just like you would test your normal elements on the page. Test isolation. Really important. So look at the test tubes here. Now imagine your test is inside a test tube and your next test is inside another test tube. Well, it's exactly like that. So nothing from one test tube will go into the other test tube, which means later on if test B fails, well, it's not because of test A because that has nothing to do with between those tests because they are running in full isolation, and that's how Playwright works out of the box.

Parallelism. Really important. Your tests run in parallel. So, it's like, imagine your tests are those swimmers and they're just running all the way until the end, right? Imagine if you had to wait for one swimmer to get to the end and then the next swimmer would go and then the next swimmer would go. That would be really, really slow. Playwright is not slow. Playwright is super fast. Works in parallel by default. With sharding, you can also scale, so imagine you're on CI and you want to run even faster, then you can basically shard across multiple machines and have your tests, say five tests run one machine, five on the other, five on the other, etc. Making your tests run even faster. Probably best not to use those machines. Should get better machines than those, but you know what I mean.

VS Code extension. I highly encourage you to use the VS Code extension because then you can write your tests, play your tests, debug your tests, open the trace viewer, the pick locator options, the code gen right from VS Code, making it a really, really nice experience.

So CodeGen. What is CodeGen? Well, this is really cool. It's a tool that's going to help you, especially like, you know, get started when writing your test to save you having to manually start typing every single line of code and not knowing what the element is, what the locator is. PlayWrite will do all that work for you. So when you go to your website, use it like a user would, start clicking, fill on all those forms and then PlayWrite will record your user action, put it into that file. If you're in VS Code it will create the file for you, put it right into VS Code and then you've got the test written out for you. Then you just have to add your assertions of course and then assert that something is visible or has count or etc. So CodeGen, super cool tool. And LocatorPicker, for those times when you need to write something yourself and you don't know what the locator is or even just you're debugging and something's not working, you're like what's going on with this element here? You can use the LocatorPicker and you can hover over the webpage and it will highlight the locator underneath and then you can basically copy that into your code and fix your problem or you know write that line that you didn't know what the locator was. This is really useful because we use role-based locators, that's what we recommend. So if you're not used to understanding the accessible roles of something, don't worry because you don't need to know that because Playwright will just give you the right accessible role. Once your code is good of course.

UiMode, my favorite tool, it is just amazing. So to use UiMode, you will have to open the terminal and you put in npx Playwright test –ui and what this does is opens this window here and it's got like all the tests on the left-hand side there and you can click and play through each of those tests. And then here in the middle, you've got a DOM snapshot – it's going to control the actions and you know what, we'll take a look at this later in a demo, but highly encourage you to use UiMode and it also comes with Watch mode which is really cool. HTML report – so you can literally like see what's going on in your test, how many passed, how many failed, how many were flaky, how many were skipped and go through the whole report and you can open and expand that out and see it in more detail too. And Trace Viewer – this is kind of similar to UiMode so again it's got that DOM snapshot, it's got everything you need to know and all those actions and this is really useful for debugging on CI. You know it works on my machine? Then you go push it to production and something fails on CI and you're like, what do I do now? Trace Viewer has the answer. You download that trace and you debug that trace and you'll figure out what exactly went on and how to fix that. GitHub Actions – so Playwright comes with a GitHub Actions out of the box which basically means you can run your test in CI without even having to do any configuration and that's really, really cool. So of course you can run Playwright on any CI provider, but we give you the GitHub Actions out of the box and it makes your life easy. So I highly recommend you check that out.

A couple of best practices or tips. The first one is literally write end-to-end tests. And I'm very serious about this. Many, many, many years ago people said end-to-end tests were slow, and you should write loads of unit tests and then a small bit of end-to-end tests. But the world has changed. The tooling has changed.

Everything has changed. End-to-end tests tooling is so much faster, it's so much cheaper than before that you don't have to write so many unit tests now. And you want your unit, you want your test to be as close to the user as possible. Usually your tests are not close to the user, they're close to the code. You want your tests to be closer to what the user is doing, so write those end-to-end tests. So if you want to call them integration tests or whatever, but make sure you're writing end-to-end tests.

If you can only have time to write one test, make it an end-to-end test. Keep PlayWrite up to date. So we release a new version of PlayWrite every month, and like, obviously we give you some really cool new features, and you're like, oh, maybe I don't need that feature, fine. But also like some bug fixes, and maybe like, I don't have any bugs, fine. But also, you're testing on the latest versions of the browsers, and you know, Chrome releases every month as well, right? So your browsers are being released every month. So by updating to PlayWrite, you're also testing against the latest browsers, and that's really, really important. So make sure you keep PlayWrite up-to-date, npm install dash d at playwrite slash test at latest.

Use locators. So locators come with the auto weighting and the retryability. And we highly encourage you to prioritize the user-facing attributes. So you'll see something like this page, get by role, what role am I looking for, I'm looking for the role of a button, and what is the name of that accessible role it is submit. And that is my user-facing attribute. I'm also testing that this button has a good accessible name. So you know, this is really good, because I'm testing accessibility. Plus I'm getting my test written and making sure my code is good. So I highly encourage you to use our playwright locators and user-facing attributes.

Use playwright assertions. So we talked about this before, they're auto retrying. So make sure you're using them. An example is a weight, make sure you have the weight, and then you expect, and you expect the page here, we're doing a get by test ID. And we've got the status, and we're expecting it to have the text submitted. So that's a playwright assertion to have text. And you'll see a whole list of those in the docs to have count, to have text, to be visible, etc.

Avoid testing third party dependencies.

So if you're going to an API, right, and maybe you don't own that API, well, you don't need to test that API, like GitHub, for example, I want my site to link to GitHub. Well, I don't need to test that. So I just need to test that the button is clickable and it will go there. But then I can kind of like intercept that route, and then just fill it with some other data. And then I know that the route is working, the link is working, but I'm not actually hitting the API each time.

Use Code Gen. Seriously, some people like stay away from tools that are like, oh, it's doing all this work for me, and I don't know what it's doing, or it's scary, I like to write all myself. I mean, totally cool if you do you want to do all that hard work yourself, it's fine. But the tools really make your life easier. So we highly encourage you to use it.

Let me give you a quick demo of how it works. So I'm in VS code. I'm just going to go record new and you'd see it's created a test one file for me. Now it gives me a browser window and I can go and test this Contoso Traders website. So it opens the browser for me on this website, and you can see here in the test, I've got a wait page, go to the cloudtesting.contostotraders.com website. Very cool. Now let's do something. Let's go and search for an Xbox. This is a user flow. What the user would do, they'd search for the product, type it in, press enter, and then search. Which one do they want? I'm going to select the one in the middle because it looks cool. And let's add two because playing alone is no fun. And then let's add it to the bag. Now up in my bag, you can see that one was added, so I can go in there. I can see that product is there. There's two in there, and basically that's, you know, really expensive. Let's remove it. I've changed my mind. I don't want to buy it. My cart is empty and my flow is done.

Perfect. Now look at my test. This has all been generated for me. Now this is really cool because I didn't have to do all that work to get that up and running. In the matter of a minute, I've got my test written. I can literally just press cancel, and then I could run that test and see that it actually works. Just, you know, make sure, I don't know, something didn't go wrong.

So if I run that test, yes, it works. Of course it works. Here's my trace. I can see the actual user flow. I can go through and see if I missed out anything. And really now what I want to do is start adding my assertions in, right? So I want to say, okay, at the end, I'm going to say expect the Locator to be visible, right? Now I'm going to run this test. What's going to happen? It's going to fail, which is obvious because I press the remove button. I don't want to see this in my cart, but I like to see that my tests fail before it passes. So you know, then I'm really assured that it's working. So I'm moving that up now before they remove. It's in the cart. Then I press the remove button. What I can do is I can put that back in here. But instead of saying to be visible, I can say, I want you to not to be visible. So once I press remove, make sure that Xbox is not visible on the page. And if I run that test, it will now pass. I've got my two assertions and my test is done in a matter of seconds, and I'm happy and ready to go home. So that's really, really cool.

So yes, use Code Jam. It's amazing. And also use UI mode. Let's take a look at UI mode. It's my favorite tool.

So we talked about it a little bit earlier. So here it is, it's launched and I can press play. Once I press play, it's going to run that test for me. And it's giving me everything that I need, right?

This is my other test. I can run that. I can see the actions. This is a very simple test where I'm just clicking the getting started button, and it's going to the installation page. And you can see the code being highlighted there at the bottom. And you can see the timeline being highlighted.

Let's run a more a better test. The test we just created with CodeGen and here we've got our test. So you can see I can go through each of those actions in the UI mode. I can see what's being clicked on. I can see the source code underneath and I can go through each step and understand better what's happening. This is really, really good for debugging, because something's going to happen in a second. You're going to watch, so wait for a second and you'll see what's going to happen.

Right? I've got my camera box and like something has happened, right? You can see now I've got an error message. The test is timed out, so in that space of like while I was talking to you, my test timed out and I've got to figure out why. What happened? We wrote a test a minute ago. What just happened here? Did the developer change the code? I hope not. So let's have a look. Well, we've got an error tab here and it's, well, the source code first tells me line 12 get by label bag dot click. So it's waiting for the get by label of bag. The error tab will tell me this as well. It's waiting for get by label bag. So that's all. That's good. You know, I can see it fails, but what do I do from here? And I can go through, you know, each of the sections, the call. I can see the locater. Strict mode is true.

So that's fine. That's not really helping me here. I can also go through my whole network requests as well, which is really cool just in case you needed to do that. I've got no console messages. That's good. But I do have an error message and I need to fix this.

Now, what would you do in this situation? You've got a fail test. How do you fix your fail test? What do you do? So the best way to do it is try and understand the flow of what was happening to that point. Right? We had a page, get by button name, add to bag, and we clicked it. So this is where we're going wrong. We've added to the bag, but then we went and clicked the bag. So we can see the steps, right? We can see what happened before. We can see what happened after. You can see the little one is added to the bag. So our product was added to the bag, but for some reason, it's not letting us click it. Now, why can playwright not click that button? This is what we're trying to figure out. And you can see in the timeline here, I can kind of scroll across there and also see the actions. I can see again, yeah, that one was added to the bag.

So this is really cool, but I don't know. How do I fix this? Well, there's a couple of ways we can do this, right? We can basically say that this is our problem. Let's put it in a DOM snapshot. Let's get it out there. Let's understand the code. Let's see what happened. Did the developer change something? Because it's a DOM snapshot of our test, we can actually inspect. We can go to the code. And if we look at this here, we can say, oh my gosh, look at that. There is an area label of cart. So some developer has gone and changed this to cart. And you can even look at this through the accessibility tree as well.

And you can see it's button cart, area label cart. So this is our problem, right? We have in our locator, bag, get by label bag. But in our code, we have cart, very simple mismatch because it looks like a bag to me. So as a tester, I think that sounds good. But obviously, the developer thought it was a cart.

Now, if I use the pick locator button even easier, I can just actually hover over that and it's going to give me the right locator and I can click it. It's going to put it in that box below. So I don't even have to look at the DOM. I don't even have to look at the accessibility tree. I can just use a pick locator and just hover over it. And I can actually play around here as well. I can write, if I put cart, if I put bag, nothing is found, right? So it can really see what's going on and just fully play around with your code here with the locator playground. So this is a great way. So we figured out get by label cart is the way to go. And you can even experiment even more. Let's get this one here. Get my role button name add to bag. Let's remove add to bag and just put add. You can see it still finds it, right, because it's got add. Even though it's got add to a bag, it's not an exact match. So be careful with that. But the locator is going to help you. This tool is going to help you figure out, oh, add to bag. There's only one of one, et cetera. So that's my pick locator. So let's go back and get that one we want. We can copy that now into our code. Now, before we copy it, let's just click Watch Mode, because we want to have this rerun automatically. And let's click the File icon, which will then just open our VS code for us. We'll go to line 12.

We'll paste in our locator, the correct one. And automatically, it's going to rerun that test for us. And you'll see here, it's rerunning. It's going to go through this. And now Playwright can find that locator, because that actually exists in the page, and happy days. Our test passes. And you can see now that it's working exactly as it should do. So that's how we would debug, et cetera. Really, really cool UI mode, so make sure you're using UI mode during your whole developer experience.

So scaling. We talked a little bit about scaling. Basically, you want to run on CI. We highly recommend you run your tests on CI. And as I said, it comes with a GitHub Actions workflow. So that just comes and works out of the box. So make sure you use that. And then if you want to shard, you can also merge reports. So here I'm taking my website. And if I go back here, my test duration was 18 minutes, right? And then just by sharding on GitHub, I got it down to eight minutes. So I run it across four machines. And then I was able to merge those reports to have one report, which gives me the one report with my 292 tests. And one was skipped. I need to fix that. 291 passed, and we're talking about eight minutes. That's not bad at all. So that's merge reports.

So remember, Playwright is open source and free. It's by Microsoft. And you can see that we are literally growing continuously. We love GitHub stars.

Make sure you star us on GitHub. We get paid in stars. So if you don't star us, we don't get paid. I know, I'm only messing. But yes, do give us a star if you like what we're doing.

We have an amazing ambassadors who are doing a crazy, cool work out there. So make sure you're following them and seeing what they're doing. And follow us on Discord, where you're going to see, not just videos and articles, but also a really helpful community for helping you with the Playwright help. So ask any questions in there, and the community got your back.

So my only question is, are you ready to Playwright? I hope you are. I hope you can now see that Playwright is easy, it's fun. Testing is easy, is fun. And now it's all down to you. I can come back here next year and give this exact same talk. And we're going to go nowhere. You need to do the work.

So make sure you get out there and just start writing your tests. And it takes a second, start small, get one done, five minutes a day. Five minutes a week, even. Oh, my gosh, you can imagine what you'd achieve in a year if you just dedicated five minutes a week to writing a test. Thank you very much, everyone. Have a great day.