Advanced Playwright Techniques for Flawless Testing

Hi everyone, Advanced Playwright Techniques for Flawless Testing. My name is Devi O'Brien, and I'm a Principal Technical PM at Microsoft. I just got promoted, and I'm basically focusing on playwright and ensuring that the community around playwright exists is amazing and that there's amazing content so you can all learn testing, because the goal is that everybody should be testing. I'm an international speaker, teacher, YouTuber, Open Source Contributor, and I absolutely love sport. I just got the OK to go back running after having twins. So yeah, follow me if you want to see lots of pictures of me running and cycling around the island.

But let's go and talk about playwright. So what is playwright? For those that do not know, playwright is reliable end-to-end testing for modern web apps, and it works in any browser, any platform with one API. Tests run in full isolation, so there's no need to clean up, and one doesn't go from one test to the other. Everything is just isolated. That's really cool and makes it much easier.

And tests run really fast. What about fast these days, right? Well, the tests run fast. That's really cool. We've got some powerful tooling. We can do things like use a code gen to just record your test by just playing around with the website, lots of cool stuff. And it's multi-language as well, so you can use JavaScript, TypeScript, or you could use Python and Java and .NET.

So why should you use playwright? Well, playwright auto-waits for the elements, so if you're using playwright correctly, basically it's going to wait for the elements to appear on the page. It also retries assertions. You can use iframes, Shadow DOM. As I mentioned before, the tests are isolated, and then tests run in parallel by default, and that's what makes it much quicker. You can also shard. If you're into scaling, you need to shard across multiple machines. That just works. And then, of course, our tooling, we have the VS Code extension. We've got code gen for generating your code. We've got a locator picker to find the locator. A locator is the element on the page, right, the button. So find that locator. We've got UI mode. I'll probably show you some UI mode stuff today. And then we've got HTML reports, Trace Viewer, and GitHub Actions. It just works out of the box with GitHub Actions, but you, of course, you could use any CI provider.

So let's dive into some tips, API mocking, testing date and time, setup tests, and a little bit of the CLI. Let's jump in. So a couple of tips, really, really important.

Use locators. So locators have the auto-waiting and retryability, so it's really, really, really important to use the proper locators. That means try not, as much as possible, do not use CSS locators. Definitely don't use XPath. So to find an element on the page, use what we call locators. And we have user-facing attributes. We want to prioritize that. So we have things like GetByRole. For example, GetByRoleButton, with the name submit, is much more user-facing. It's much better for the user. User sees it, right, as opposed to GetByTestID. You could use GetByTestID, but we recommend user-facing attributes first.

Use locators, auto-waiting and retryability. Use playwrights assertions. So instead of saying to be, use the playwright assertions to have text, to be visible, et cetera, because these are auto-retrying and that will ensure less flakiness in your test. So here I've got an example. I'm using a test ID in this example, expectPage.GetByTestID of status to have the text submitted.

Annotations. This is really cool. Are you using annotations? This is a playwright report. And when you use annotations, you get the annotations in the report. You also get it in UI mode or in the trace viewer. So here in the annotations, I've got like an issue. I've got related issues. So there could be something going on with this test and you want that information to be seen in a report. And this is the perfect place to do it, so use annotations. Now, to add an annotation, all you've got to do is just put the annotation just straight after the test name. So I call the test Use Annotations. Terrible name. But test could be whatever.

And then just open up those curly brackets and write an annotation. Annotation is an array, so it accepts more than one type. And you can put your own type or description. You could say type Debbie, right? Whatever you want. So that's really cool and they're clickable as well.

So in the report, you just click and go straight to that issue. And it doesn't have to be issues. It could be whatever you want. So here's a little kind of idea I've created just to show you what it looks like. So here I've got a test. I'm going to run this test. And what is going to happen is it's going to run all the tests in the manage list. Now, I've got one test here. You can see it's kind of not running. So why is that? Let's have a look at the annotations tab. And that's because I've put a fix me. So fix me is also called an annotation. So there's something wrong with this test. I need somebody to fix it, so I'm going to put fix me.

Now, the last test here, it's kind of getting a little bit slow. But look, it's fail. I've actually got a test.fail there. So the annotation is coming as fail. And I've got the issue to show anyone who's running this test that there's an issue. This test is going to fail until someone fixes that issue. And you can see here I've got test.fail. That means it's not going to mess up CI. But the test will fail. I want it to fail. And I've got my annotation in there so we can keep a check on that.

And then filter by fail test later on, et cetera. So annotations are very cool. Test steps. Test steps are really nice, especially if you have a really long test. You want to just put certain things into a little step. So here's a test step dot create first to do. And then I fill that up, and I do that kind of stuff. And then I got another step, create the second to do. And of course, this could be a function. But this is an example of, like, I have it in my movies, one where I've just got a lot of stuff going on, and I split it up into test steps. Makes it much easier to see in the UI mode and also much easier in the Trace Viewer and in the report itself. So here in the report, for example, I just go create first to do, then expect top text. Create second to do, bam. And I don't really care about the implementation details, but I can just, you know, open that up if I really wanted to and see that more. So it's just a nice kind of way of doing that. So I encourage you to use test steps. And of course, you can box the test steps as well. So you could say box dot true. And what this does is it basically just stops the error message from, like, giving you too much information. So here I've got, like, the error message, the new to do dot fill, and I don't know where to go, right? There's two little errors there. But with the box steps, it's basically saying, the error is in the get by placeholder, what needs to be done. So I can go kind of straight to that area to kind of fix that rather than kind of focusing on too much. So that's another option as well, boxing your steps.

Okay, let's go into API mocking. API mocking is quite fun. And there's a lot of kind of, like, should we mock our APIs and stuff? And obviously, you want to run into and test. You want to test everything as much as possible. But there are times when you will need to mock your API, and that's totally okay. So I'm going to show you by doing a demo, I'm going to use the UI mode. In case you do not know, npx playwright test dash dash UI is what's going to open up the UI mode.

So once I do that, I have the UI mode open, and all my tests are here. Now I've got this test on the movie details page. So here's my movie details, right? Now this movie is not going to change much. It's very static, right? The title isn't going to change, the genre isn't going to change, the description, the characters. I mean, they shouldn't change, right? The movie is finished. It's a finished product. But the rating is going to change. And let's just pop that out so you can see it closer, right? I've got a rating of 7.02. Now that rating is constant, so tomorrow, that could be 7.03, right? So how do we test that dynamic content as part of this static page? And that's where you kind of want to mock the API a little bit, because really, you just want to make sure the stars show up in the page. So how would we do that? Well, basically, in our test here, I've got a waitPage.root there in line eight, and I'm just putting in the root that we would go to, which is the ID of the movie. So when I hit the root of the ID of that movie, I want you to go and fetch that Json, right? Wait for that Json to come back. And then I want to say, well, Json, I want the vote average to be 7.02. Okay, now that's the same as what it is. That's fine. Don't worry about that for now. And then I'm just asserting that it's actually there, right? Now, if I go into my network tab in the UI mode, I can actually have a look, and I can see in the body of this, the 7.02, right? That's correct. So that doesn't make much sense to you, because you're like, why are you mocking something that's exactly the same as it is, right? I can see the API, the API body is also the same. So there's not really much difference here, okay? But what we want to do is let's change it. So let's imagine, I don't know, everyone watched that film, they all thought it was amazing, and we got a 10, right? So 10 came back from the API, but my static content, my test is going to break because it's going to be looking for 7.02, but something changed in the API. I got a 10, my test failed. Look, the 10 is shown there. That's great. I can see that everyone loves this movie, but everything failed. So how do I fix that? And that's basically where you want to mock that API. And if I look at my fulfilled, see the way it shows me that this is the fulfilled, I can go to the body and it's going to show me in the API, in the JSON, the vote average is 10, because that's what I mocked, okay? That's what I intercepted, I went and changed that. And you can see my test is failing because it's expecting the 7.02, which is in the test. And of course, I can go back to the test and change that to 10, and I could leave the vote average of 10 if I wanted to. And I've got obviously Watch Mode on there, so that test just runs automatically. And my test passed, and I've got 10. So it doesn't really matter to me what the vote is.

What matters to me is that everything works as it should. So that's how you do that. Now you can also, basically, go to a route, for example, on this website, I don't want to hit the IMDB website every time that I want to test the site, but I want to make sure that the link is clickable. So you can see here, every movie clicks to the website, it clicks to the IMD website, and I'm basically just saying, when you want to go to that website, just go here instead. So mocking, right?

I'm mocking that, so that it doesn't hit that website, but I can still test that that button is clickable and it's going to go to where it should go. Let's take a quick look at how you would do that. So in the same test, I actually have this in the same test, look how I'm using test.stats, you see? My beautiful tests, and amazing. So here, right? I've got my page, wait, page.context. Look at that line 77, and I'm saying route. So the Twister's movie website. So when you go to the Twister movie's website, I'm just going to create an HTML page and mock page, right? To just to prove that I'm going to that page. And then I basically just, and the same here, right? For the IMDb website, every time you see that route, intercept it, and fulfill it with this body instead, this mock thing that I'm creating. And then I basically get by the little button, click it, and then I wait for the page one to have the URL, IMDb. So that's kind of a way of mocking external links that you want to go to, but you want to make sure that they are clickable and that somebody can do that. So that's really cool, and I can just play that again, and you can see how it basically just goes and finds the Twister's movie website instead of actually going to the actual website. You can see www.twistersmovie.com, right? So it's really, really cool. So yeah, that is API mocking.

So let's take a look at testing date and time. How would you do that? So you might want to test something like this. We've got a clock, right? And we've got 16, 36, 52, and it's kind of counting upwards, and it's using date.now. So basically, what I've got to do, I've got a function called startClock. I've got the const today, my date, etc. And I've got a window addEventListener, load, start the clock, right? So that's basically my clock, HTML. And then my test, what it's going to do is I'm basically using the page.clock, and I'm setting a fixed time. So my fixed time is a new date at 10 o'clock in the morning on this specific date. And then I go to my clock website. So fixed, setFixedTime returns a fixed fake time at all times, okay? And then I basically run this test, and I'm just making sure that the time is 10 o'clock. I've got that passes, and you can see in my trace viewer here, I'm running through the VS Code extension, and I get the trace viewer because I have that set. And it just shows, yep, it's going to 10 o'clock. So perfect.

Really, really easy to test. And you can see there that the time was actually 4.37. It is not 10 o'clock, but I have like mocked that time. So that's how you can kind of stop time and change the time and go back and forth in time. Very cool.

What about manually advancing the time? So you can kind of use clock install to initialize the clock. And you can say, I want you to initialize at this specific time. So these fake timers are used to manually control the flow of time and tests. And then you can advance it. So then you can say, clock.pauseAt, for example. So pause it at 10 o'clock, and then you can expect something to happen. And then you can fast forward at 30 minutes and expect something else to happen, right?

Something else, I've got a countdown, right? So you know those offers, right? I've just refreshed the page there, so you can see that countdown, it's flash off. You've got five minutes to purchase. How would you test something like this without having to wait five minutes? So you'd do the clock.install, and then basically expect the flash offer to be visible, right? So you've initialized the clock, and then you're expecting something to be visible. And then you fast forward by five minutes because then the flash offer is gonna be finished, and then you'll expect what happens on that page. You're gonna expect the offer expire to be visible. So that's how you run that test.

Amazon did this to me the other day, five minutes to purchase, and I was like freaking out, like, ah, and I went, you know, do I buy it? And I didn't. And then I refreshed the page and it came back again. So I was like, cool. So how would you test something like this without having to wait five minutes? So you'd do the clock.install, and then basically expect the flash offer to be visible, right? So you've initialized the clock, and then you're expecting something to be visible. And then you fast forward by five minutes because then the flash offer is gonna be finished, and then you'll expect what happens on that page. You're gonna expect the offer expire to be visible. So that's how you run that test.

Let's have a look and see it in action. And my test passes and you can see there my clock initialized, I've got five countdown, and then I see the flash offers there, and then I basically fast forward and I get my offer expired. So that's how you test something like that. Again, make sure it's using things like data now and not as something coming from the backend.

Cool, what about setup tests? So what about when you have something like a login scenario, something that you wanna do before other tests? So I've got a demo to show you this. Again, mpxplaywritetest-ui to open up that UI mode so that you can kind of run your tests and play around with things a lot easier. So what we're gonna do here is I have a movies site where I need to log in to manage the list of movies. So my movie site needs to be logged in and I can basically look up here in the UI mode, because I've got UI mode open. If I click this little thing, I can click on the setup test. I wanna show, I wanna see that. So that's hidden for you. You can click that to basically see it. So I've got my setup test and I go and run a watch. I just run the add and delete a movie test, but it automatically ran the log the user in test and then it went and run the movies test. So it depended on it. The setup was run first. And then I can quickly run along. This is a really long test about adding a movie into my movie list so that I can create the movies that I wanna watch, et cetera. So here is like, again, really, really cool test. See how the UI mode, you can kind of really easily kind of see anything. You can click something there and expand that timeline, which is great for kind of debugging and checking things out. My source code is down below and you got everything else easily at hand to you. So how does that setup work? Well, basically let's go back to VS Code and you can see here my login.setup.ts file. Basically I'm using import test a setup so that I can call it setup, so it makes it clearer for me. Setup uses the logs in. And then basically that setup test is in my project, right? So it's a project named setup and my logged in Chrome test or project, every test on that project depends on setup and then it uses the storage state, right? The login is just logging into the simple test, but everything is important about the project. The project dependencies, this is called. Now I can uncheck in UI mode, that setup and I can run that test again. And now it's not running the logged in test because it already has it saved in storage state. And that makes it really quickly for in development.

You don't need to be constantly running that log in test all the time. It's run at once, log in, uncheck it and then just keep running your tests that need the login, but already has it logged in because you've already run it once and it's saved it into that storage state. And that is project dependency. So highly encourage you to check that out. It doesn't have to be logged in, it could be anything. You could be creating a movie list, could be one that depends on the, I don't know, whatever. But lots of really cool things you can do with the project dependencies.

A little couple of things extra, we've got last failed for the CLI. So if you're not using the VS code extension, you're using the CLI, this is kind of really cool. You can basically run the last failed test. So instead of running 103 tests using five workers where two failed and 101 passed, you can basically do npx playwrite test dash dash last failed and it will only run those last two tests that failed and see then if they, when you worked on them, have they been fixed or they're still failing. That's a really kind of quick workaround. Also only changed for the CLI. So you basically would only change, you can only run, you only run the test files that have been changed since the last comment.

So here, like for example, git status, I'm on the branch main. Change is not staged for comment, our tests only changed. This is my test is called only changed example. And then when I run npx playwright test dash dash only changed, I run one test using one worker and it runs that test. This is really cool. And what's next? Well basically playwright is open source and free. So make sure you are using playwright. Look at all their stars, right? We love stars. We're climbing high, but we still want more stars. So stars on GitHub, we really appreciate that. NPM downloads is true to the roof. It's playwright, it's by Microsoft. So you have absolutely no excuse to not be using playwright. And we have amazing ambassadors. I highly encourage you to follow all of these people. They're amazing and they're creating some really great content out there.

You can find the ambassador page under the community tab of our website. And don't forget to join our playwright discord. We have 14,000 members in there. That is insane, I'm so impressed. And there's constant people helping each other out, sharing content. And we also do happy hours in there. So come and join us on the playwright discord stage.

And of course, are you ready to playwright? I hope you all enjoyed this. I hope it helped you. If you're new to playwright, make sure you just come on board, just try it out. If you're an advanced user, make sure you're in a discord server and make sure you're chatting to other community members and creating issues for anything that you have that you're not unsure of.

And basically, thank you very much. I hope you enjoyed the talk. Remember to read the docs, the docs are really important. I know you're not reading them, you should read them. But also you can also check our YouTube channel where we kind of post some cool videos and things like that. Our discord channel, as I mentioned, we're on Twitter. And if you need anything, you just send me an email, debiobrian at microsoft.com or check out my website debiobrian.codes. Thank you very much, everyone for listening and have a great rest of the talks. Bye.