Advanced Playwright Techniques for Flawless Testing

Hi, everyone. Advanced Playwright techniques for flawless testing. My name is Debbie O'Brien and I'm a Principal Technical PM at Microsoft. I just got promoted and I'm basically focusing on playwright and ensuring that the community around playwright exists is amazing and that there's amazing content so that you can all learn testing because the goal is that everybody should be testing.

I'm an international speaker, teacher, YouTuber, Open Source Contributor, and I absolutely love sport. I just got the OK to go back running after having twins. So, yeah, follow me if you want to see lots of pictures of me running and cycling around the island. But let's go and talk about playwright.

So what is playwright? For those that do not know, playwright is a reliable end-to-end testing for modern web apps, and it works on any browser, any platform with one API. Tests run in full isolation, so there's no need to clean up and one doesn't go from one test to the other. Everything is just isolated. That's really cool and makes it much easier. And tests run really fast. What about fast these days, right? Well, the tests run fast, so that's really cool. We've got some powerful tooling. We can do things like, you know, use a code gen to just record your test by just, like, playing around with the website. Lots of cool stuff, and it's multi-language as well. So you can use JavaScript, TypeScript or you could use Python, Java and .NET.

So why should you use playwright? Well, playwright auto-waits for the elements. So if you're using playwright correctly, basically, it's going to wait for the elements to appear on the page. It also retries assertions. You can use iFrames, Shadow DOM. As I mentioned before, the tests are isolated and then tests run in parallel by default, and that's what makes it much quicker. You can also shard. If you're into scaling, you need to shard across multiple machines, that just works. And then, of course, our tooling, we have the VS Code extension. We've got CodeGen for generating your code. We've got a locator picker to find the locator. A locator is the element on the page, right, the button. So find that locator.

We've got UI mode. I'll probably show you some UI mode stuff today. And then we've got HTML reports, trace viewer, and GitHub Actions. It just works out of the box with GitHub Actions. But you, of course, you could use any CI provider.

So let's dive into some tips. API mocking, testing date and time, setup tests, and a little bit of the CLI. Let's jump in.

A couple of tips, really, really important. Use locators. So locators have the auto-waiting and retryability. So it's really, really, really important to use the proper locators. That means try not, as much as possible, do not use CSS locators. Definitely, don't use XPath. So to find an element on the page, use what we call locators.

And we have user-facing attributes. We want to prioritize that. So we have things like getByRole. For example, getByRole of button with the name submit is much more user-facing. It's much better for the user. User sees it, right, as opposed to getByTestID. You could use getByTestID, but we recommend user-facing attributes first. So yes, use locators, auto-waiting and retryability. Use playwright's assertions. So instead of saying to be, use the playwright assertions to have text, to be visible, etc., because these are auto-retrying, and that will ensure less flakiness in your tests.

So here I've got an example. I'm using a test ID in this example. Expect page.getByTestID of status to have the text submitted. Annotations. This is really cool.

Are you using annotations? This is a playwright report, and when you use annotations, you get the annotations in the report. You also get it in UI mode in the trace viewer. So here in the annotations, I've got an issue. I've got related issues. So there could be something going on with this test, and you want that information to be seen in a report. And this is the perfect place to do it. So use annotations.

Now, to add an annotation, all you've got to do is just put the annotation just straight after the test name. So I call the test use annotations. Terrible name, but test could be whatever. And then just open up those curly brackets and write an annotation. Annotation is an array, so it accepts more than one type. And you can put your own type or description, right? You could say type Debbie, right? Whatever you want. So that's really cool, and they're clickable as well.

So in the report, you just click and go straight to that issue. And it doesn't have to be issues. It could be whatever you want. So here's a little kind of idea I've created just to show you what it looks like. So here, I've got a test. I'm going to run this test. And what is going to happen is it's going to run all the tests in the manage list. Now, I've got one test here. You can see it's kind of not running. So why is that? Let's have a look at the annotations tab. And that's because I've put a Fix Me. So Fix Me is also called an annotation. So there's something wrong with this test. I need somebody to fix it. So I'm going to put Fix Me. Now, the last test here, it's kind of getting a little bit slow, but look, it's fail.

I've actually got a test.fail there. So the annotation is coming as fail. And I've got the issue to show anyone who's running this test that there's an issue. This test is going to fail until someone fixes that issue. And you can see here, I've got test.fail. And that means it's not going to mess up CI, but the test will fail. I want it to fail. And I've got my annotation in there so we can keep a check on that and then filter by fail tests later on, et cetera. So annotations are very cool.

Test steps. Test steps are really nice, especially if you have a really long test. You want to just put certain things into a little step. So here's a test step.createFirstToDo. And then I fill that up and I do that kind of stuff. And then I got another step, create the second to do. And of course, this could be a function, but this is an example of, I have it in my movies one where I've just got a lot of stuff going on and I split it up into test steps. Makes it much easier to see in the UI mode and also much easier in the trace viewer and in the report itself. So here in the report, for example, I just go createFirstToDo, then expect top text, create second to do, bam. And I don't really care about the implementation details, but I can just open that up if I really wanted to and see that more. So it's just a nice kind of way of doing that. So I encourage you to use test steps. And of course, you can box the test steps as well. So you could say box.true. And what this does is it basically just stops the error message from giving you too much information. So here I've got the error message, the new todo.fill, and I don't know where to go, right? There's two little errors there. But with the boxed steps, it's basically saying the errors in the getByPlaceholder, what needs to be done. So I can go kind of straight to that area to kind of fix that rather than focusing on too much. So that's another option as well, boxing your steps.

Okay, let's get into API mocking. API mocking is quite fun. And there's a lot of kind of like should we mock our APIs and stuff. And obviously you want to run end-to-end test. You want to test everything as much as possible. But there are times when you will need to mock your API and that's totally okay. So I'm going to show you by doing a demo, I'm going to use the UI mode.

In case you do not know, NPX Playwright test -- UI is what's going to open up the UI mode. So once I do that, I have the UI mode open and all my tests are here. Now I've got this test on the movie details page. So here's my movie details, right? Now this movie is not going to change much. It's very static, right? The title isn't going to change, the generator isn't going to change, the description, the characters. I mean, they shouldn't change, right? The movie is finished. It's a finished product, but the rating is going to change.

And let's just pop that out so you can see it closer. I've got a rating of 7.02. Now that rating is constant. So tomorrow that could be 7.03, right? So how do we test that dynamic content as part of this static page? And that's where you kind of want to mock the API a little bit because really you just want to make sure the stars show up in the page. So how would we do that? Well, basically in our test here, I've got a wait page.root there on line eight, and I'm just putting in the root that we would go to, which is the ID of the movie. So when I hit the root of the ID of that movie, I want you to go and fetch that JSON, right? Wait for that JSON to come back. And then I want to say, well, Jason, I want the vote average to be 7.02. Okay, now that's the same as what it is. All right, that's fine. Don't worry about that for now. And then I'm just asserting that it's actually there, right? Now, if I go into my network tab in the UI mode, I can actually have a look and I can see in the body of this the 7.02, right, that's correct. So that doesn't make much sense to you because you're like, why are you mocking something that's exactly the same as it is, right? I can see the API. The API body is also the same. So there's not really much difference here, okay? But what we want to do is let's change it. So let's imagine, I don't know, everyone watched that film, they all thought it was amazing and we got a 10, right? So 10 came back from the API with my static content. My test is going to break because it's going to be looking for 7.02, but something changed in the API.

I got a 10, my test failed. Look, the 10 is shown there. That's great. I can see that, you know, everyone loves this movie, but everything failed. So how do I fix that? And that's basically where you want to mock that API. And if I look at my fulfilled, see the way it shows me that this is the fulfilled, I can go to the body and it's going to show me in the API, in the JSON, the vote average is 10 because that's what I mocked, okay? That's what I intercepted, I went change that. And you can see my test is failing because it's expecting the 7.02, which is in the test. And of course I can go back to the test and change that to 10 and I can leave the vote average of 10 if I wanted to. And I've got obviously watch mode on there. So that test just runs automatically and my test passed and I've got 10. So it doesn't really matter to me what the vote is. What matters is to me is that everything works as it should. So that's how you do that.

Now you can also in like basically go to a route, for example, on this website, I don't want to hit the IMDB website every time that I want to test the site, but I want to make sure that the link is clickable. So you can see here, every movie clicks to the website, it clicks to the IMD website. And I'm basically just saying, when you want to go to that website, just go here instead. So mocking, right, I'm mocking that so that it doesn't hit that website, but I can still test that that button is clickable and it's gonna go to where it should go. Let's take a quick look at how you would do that. So in the same test, actually I have this in the same test. Look, I'm using test.stabs. You see my beautiful tests and amazing. So here, right, I've got my page. I wait page.context, look at that line 77 and I'm saying route. So the Twister's movie website. So when you go to the Twister movies website, I'm just gonna create an HTML page and a mock page, right, to just to prove that I'm going to that page. And then I basically just, and the same here, right, for the IMDB website, every time you hit, see that route, intercept it and fulfill it with this body instead, this mock thing that I'm creating. And then I basically get by the role button, click it, and then I wait for the page one to have the URL, IMDB. So that's kind of a way of mocking external links that you wanna go to be wanna make sure that they are clickable and that somebody can do that. So that's really cool. I can just play that again and you can see how it basically just goes and finds the Twister's movie website instead of actually going to the actual website.

You can see www.twistersmovie.com, right? So it's really, really cool. So yeah, that is API mocking. So let's take a look at testing date and time. How would you do that? So you might want to test something like this. We've got a clock, right, and we've got 16, 36, 52, and it's kind of counting upwards, and it's using date.now. So basically what I've gotta do, I've got a function called startClock, I've got the const today, my date, et cetera, and I've got a window addEventListener, load, start the clock, right?

So that's basically my clock, HTML, and then my test, what it's gonna do is I'm basically using the page.clock, and I'm setting a fixed time. So my fixed time is a new date, at 10 o'clock in the morning on this specific date, and then I go to my clock website. So setFixedTime returns a fixed fake time at all times, okay? And then I basically run this test, and I'm just making sure that the time is 10 o'clock, that passes, and you can see in my trace here, I'm running through the VS Code extension, and I get the trace view because I have that set, and it just shows, yep, it's going to 10 o'clock. So perfect, really, really easy to test. And you can see there that the time was actually 4.37, it is not 10 o'clock, but I have mocked that time. So that's how you can kind of stop time, and change the time, and go back and forth in time, very cool.

What about manually advancing the time? So you can kind of use clockInstall to initialize the clock, and you can say I want you to initialize at this specific time. So these fake timers are used to manually control the flow of time in tests, and you can advance it. So then you can say clock.pauseAt, for example, so pause it at 10 o'clock, and then you can expect something to happen. And then you can fast forward it 30 minutes, and expect something else to happen, right?

So this is a very simple test, but you could imagine what you'd be doing there during those to make sure it's at 10, and it's then at another time, and then it's fast forwarding. So let's just run that test so you can quickly see how that works super fast, and you've got like eight o'clock, and I've got 10 o'clock, and then I'm fast forwarding, and then I've got 10.30, so very, very cool. And then it stops you having to wait 30 minutes, right?

You've got five minutes to purchase. Amazon did this to me the other day, five minutes to purchase, and I was like freaking out, and I went, you know, do I buy it? And I didn't, and then I refreshed the page, and it came back again, so like cool. So how would you test something like this without having to wait five minutes?

So you'd do the clock.install, and then basically expect the flash offer to be visible, right, so you've initialized the clock, and then you're expecting something to be visible, and then you fast forward by five minutes because then the flash offer is gonna be finished, and then you'll expect what happens on that page. You're gonna expect the offer expired to be visible. So that's how you run that test.

Let's have a look and see it in action. And my test passes, and you can see there my clock initialized. I've got five, countdown, and then I see the flash offers there, and then I basically fast forward, and I get my offer expired. So that's how you test something like that. Again, make sure it's using things like date.now and not something coming from the backend.

Cool, what about set up tests? So what about when you have something like a login scenario, something that you wanna do before other tests? So I've got a demo to show you this. Again, mpxplaywritetest-ui to open up that UI mode so that you can run your tests and play around with things a lot easier. So what we're gonna do here is I have a movies site where I need to log in to manage the list of movies. So my movie site needs to be logged in, and I can basically look up here in the UI mode because I've got UI mode open.

If I click this little thing, I can click on the setup test. I wanna show, I wanna see that. So that's hidden from it for you. You can click that to basically see it. So I've got my setup test, and I go and run. Now watch, I just run the add and delete a movie test, but it automatically ran the log in, the log the user in test, and then it went and ran the movies test. So it depended on it. The setup was run first. And then I can quickly run along. This is the really long test about adding a movie into my movie list so that I can create the movies that I wanna watch, et cetera.

So here is, again, really, really cool test. See how the UI mode, you can really easily see anything. You can click something there and expand that timeline, which is great for debugging and checking things out. My source code is down below. You've got everything else easily at hand to you. So how does that setup work? Well, basically, let's go back to VS Code, and you can see here my login.setup.ts file.

So how does that setup work? Well, basically, let's go back to VS Code, and you can see here my login.setup.ts file. Basically, I'm using import test as setup so that I can call it setup so it makes it clear for me. Setup uses the logs in, and then basically that setup test is in my project. So it's a project named setup, and my logged in Chrome test, our project, every test under that project depends on setup, and then it uses the storage state. The login is just logging in as a simple test, but everything is important about the project, the project dependencies this is called. Now, I can uncheck in UI mode, that setup, and I can run that test again, and now it's not running the logged in test because it already has it saved in storage state. And that makes it really quickly for in development. You don't need to be constantly running that login test all the time, it's run it once, log in, uncheck it, and then just keep running your tests that need the login, but already has it logged in because you've already run it once and it's saved into that storage state. And that is project dependency. So highly encourage you to check that out. It doesn't have to be log in, it could be anything. It could be creating a movie list, could be one that depends on, I don't know, whatever, but lots of really cool things you can do with the project dependencies.

So a little couple of things extra, we got last failed for the CLI. So if you're not using the VS Code extension, you're using the CLI, this is kind of really cool. You can basically run the last failed test. So instead of running 103 tests using five workers where two failed and 101 passed, you can basically do npx playwright test dash dash last failed and it will only run those last two tests that failed and see then when you worked on them, have they been fixed or they're still failing? That's a really kind of quick workaround.

Also only changed for the CLI. So you basically would only change, you can only run, you only run the test files that have been changed since the last comment. So here, like, for example, get status, I'm on the branch main, changes not stage for comment, our tests only changed. This is my test is called only change example. And then when I run npx playwright test dash dash only changed, I run one test using one worker, and it runs that test. So this is really cool.

And what's next? Well, basically playwright is open source and free. So make sure you are using playwright. Look at all their stars, right? We love stars. We're climbing high, but we still want more stars. So stars on GitHub, we really appreciate that. NPM downloads is true the roof. It's playwright, it's by Microsoft. So you have absolutely no excuse to not be using playwright. And we have amazing ambassadors. I highly encourage you to follow all of these people. They're amazing and they're creating some really great content out there. You can find the ambassador page under the community tab of our website. And don't forget to join our playwright Discord. We have 14,000 members in there. That is insane. I'm so impressed. And there's constant people helping each other out, sharing content. And we also do happy hours in there. So come and join us on the playwright Discord stage. And of course, are you ready to playwright? I hope you all enjoyed this.

I hope it helped you. If you're new to playwright, make sure you just come on board, just try it out. If you're an advanced user, make sure you're on a Discord server and make sure you're chatting to other community members and creating issues for anything that you have that you're not unsure of.

And basically, thank you very much. I hope you enjoyed the talk. Remember to read the docs. The docs are really important. I know you're not reading them, you should read them. But also, you can also check our YouTube channel where we kind of post some cool videos and things like that. Our Discord channel, as I mentioned, we're on Twitter.

And if you need anything, you just send me an email, debbiobrian at Microsoft.com or check out my website, debbiop.codes. Thank you very much everyone for listening and have a great rest of the talks. Bye. Bye.