Testing Web Applications with Playwright

♪♪♪ Testing web applications with Playwright. It's time to play your tests right. Hi, everyone. My name is Debbie O'Brien. I'm a Senior Program Manager at Microsoft, and I'm basically working on advocating for Playwright. I'm an international speaker, teacher, YouTuber, open-source contributor, and you can follow me on Twitter, devs underscore o'brien, to see me running and cycling and doing all sorts of sports across the island of Majorca in Spain.

So we're here to talk about testing, so let's get straight to it. Playwright testing, reliable end-to-end testing for modern web apps, works on any browser. Doesn't matter what platform you're on. Any browser, any platform. One API. Tests run in full isolation. That means one test is here with a browser context, another test is here. They don't leak things to each other, right? So everything here is in full isolation from the other test here. Fast execution. We've got powerful tooling, which I'm gonna talk about a lot in this talk, and multi-language, which means you can run your tests in C Sharp, in Java, in Python, or in JavaScript and TypeScript. I'm gonna show you TypeScript today, but you can choose any of the other languages that you want.

I'm gonna show you the VSCode extension because I actually love the integration with VSCode. It's very, very cool, but don't worry, you can still use Visual Studio. You can still just use a terminal or whatever you prefer. But let me show you the extension. So look for the Playwright test for VSCode by Microsoft extension, and then basically in your command panel, just write Install Playwright, and click on that. And what you get is, now you get to select the browsers, and you can choose Chromium, Firefox, or WebKit. You can choose all of them or just one of them. And later you can configure it, so don't worry too much if you're not sure which you want. There's another option there, use JavaScript. If you don't wanna use TypeScript, it's TypeScript by default, but you can use JavaScript if you prefer. And then also the GitHub Actions. By clicking that box, we're gonna get a GitHub Actions, and it's gonna basically work out of the box without you having to configure anything. You'll be able to run your tests on GitHub, on CI in every push and pull request.

So once you've installed, you're gonna get this. You'll get that GitHub folder there with the actions inside it, and you get a test folder, when you example.spec.ts file. That will have a very small test in there, which we're gonna go through today, and then there's a test example file, which has a to-do demo app, and you can have a look at that in more detail.

And then the PlayWrite config, in case you wanted to later add or remove some of those browsers, or put testing for mobile Safari, or set up a dev server to run your application on localhost 3000 before you run your tests, et cetera. You can do all that in the config, but you very rarely need to touch the config, because everything just works.

So when you wanna run tests, you can run all tests. You can run a set of tests, a single test, and tests run in parallel. Without you having to do anything, automatically runs in parallel, which makes them super fast. So I'm gonna run the tests using the VS Code extension. Again, you could use the terminal and just put npx.playwritetest, but here on line three, you'll see this lovely green triangle. So I'm gonna click on that, and then basically, it's gonna go down through each line, and it's gonna run the test. Now, you can see it opened up a browser window there for me, and I was able to see everything that's going on. I'll just play that again. You can see super fast, right? That's because I have this show browser ticked, and you can uncheck that if you want to. If you do not wanna see the browser, if you wanna see the browser, make sure that that is checked.

So live debugging, if you have any errors in your test, I mean, you never have errors, right? But if you do have any bugs or anything, you have live debugging and error messaging in VS Code itself. So again, we have our test here in VS Code, and I'm just gonna like break this here, right? So getByText, I'm gonna run that test, it's gonna break, and it says, error, strict mode violation. So playwright's saying, Hey, look, there's one, two, three elements there, and you can have the whole log. Now, if you don't wanna read the log or don't understand the log, you could basically just set a break point, and you could run that test again, so you can kind of see it. So running it in debug mode, and we can now go through that and see what's happening. So at this point in time, playwright's saying, Hey, look, getByText, resolved to three elements. Here's one of them. Here's the other one. And down the bottom, we've got another one that has the word star inside that getting started text. So playwright doesn't know which one you want it to choose. So it's saying, Hey, this is breaking the strict violation. So you can play around and live debug this and say, Okay, what if I put get star? Get star only resolves to one, so that would be okay. That would work. Again, we can kind of like go back into here and play around and put something like, imagine we put playwright, right? We put playwright. There's like way too many, okay? Hey, there's like 12 of these.

This is gonna be really, really difficult selector to choose from. So, yeah, so basically going through this, you're able to kind of like live see what's going on in the page, what playwright's trying to select and see, and you'll be able to then, make sure that those selectors are much better and much more precise. And then you can just like continue, add more debugging, more break points, or just step through that test, stop it, play it, restart it again, again, et cetera.

Now locators, what are locators? So locators are a way to find elements in the page at any moment with built-in auto waiting and retry ability. Yes, built-in auto waiting, that means you don't have to do anything. Playwright's gonna wait automatically for that locator to be ready. You don't have to set time outs or do anything like that.

So, locators look like this. They're strict by default. So it's page.locator, data test ID equals directions.click, for example, let's say we're using IDs. Now, even better than that, let me show you something else because Playwright now has new APIs inspired by testing library. As you can see by this tweet from someone in the community, a lot of excitement there. So we've got like, you know, getByText, getByRole, getByLabel, getByPlaceholder, getByAltText and getByTitle. And just to show you that in kind of more detail you can see here. So, await page, getByLabel, username, philjohn, await page, getByLabel, password philjohn, password, await page, getByRole, button, name, sign in, and then expect the page, getByText, welcome John to be visible. So those make sure you're using our new locators, getByLabel, getByRole, getByText, et cetera.

And in case you're kinda wondering like, oh my gosh, this is like so difficult. How am I gonna, you know, know which selectors to pick? How am I gonna know which ones? We have a pick selector option in VS Code. Now if you're not using VS Code, you can also use the terminal. You can basically launch the inspector and you'll be able to do the same thing. So let me show you. So here we have our page. And as we hover over it after clicking that pick selector, we now getByRole heading. We can click that heading there. And inside now our pick selector box, we've getByRole, heading, name, installation. Now I can just like press Enter. It's gonna go into my clipboard and I can now go into my code and basically just paste that into my code. And now I've got my new selector set up and I'm able to kind of just use that. So I can go along here again and I can hover over and see getByRole link. This is name equals how to install PlayRide.

Let's check over here, getByRole link. This is lovely, getByRole button. And this is name, switch between dark and light mode. So that's the name of that one. And again, hovering over each item is gonna give us the selector that we want. So it's a nice easy way of kind of seeing what selectors are available to you and easily able to just like copy that and put that into your code and use it. So yes, pickSelectorBox there, make sure you play around with that and have a lot of fun picking your selectors.

So make sure you're using web-first assertions. We have a lot of assertions. We've got, to be checked, to be disabled, to be editable, to be empty, to be enabled, to be focused, to be hidden, to be visible, to contain text, to have attribute, to have class, to have count, to have CSS, to have ID, to have JS property, to have text, to have titles, have your own and so many more. Right? So I should be Robert, right? Yeah, so they're all your assertions. So that's what's gonna get a few that auto-weighting. So make sure you're using those assertions to make sure you get that.

Now, let me introduce you to Code Gen. Code Gen is my friend. I love Code Gen. Code Gen is amazing. Code Gen will generate tests by recording your actions. Now it's not perfect, you know, but it's an amazing way to get started and to really help you kind of ease into testing if you're new to testing especially. This is a really good way to get started. So let me show you what Code Gen does. Now, again, I'm gonna use the VS Code extension, but you could totally just use the terminal and launch up that inspector and be able to use it from there. So from VS Code, I'm gonna go into that testing sidebar, I'm gonna click record new. And what it's doing is it gives me a browser window, but at the same time, it gives me a test1spec.ts file. It's importing tests or playwright tests and it's starting off that test block. Now I can go to any URL. And I'm just gonna test the Netflix website for now. And I come along here and it's got a wait page, go to Netflix, this is great. Now I'll click on this button accept and you've got this lovely get by roll button, name accept dot click. So, fantastic.

And I can continue to carry along and do things. So I'm kind of like clicking on this logo here because I want to make sure that logo is there. The user might not click on that logo, right? But I'm just clicking on it so it records that action and I'll later change that to an assertion. Again, click on that heading and that's not normally a click event but I want to click on that so that it's there and then I'll turn that into an assertion later.

Now let's do something fun. Let's go down to the Help Center. We open up a new page here and let's click on something, let's say can't sign into Netflix. Okay, this looks great. We'll click on that title, make sure that is there, lovely, that's perfect. And was this article helpful? Let's click yes. And we'll just check the thanks for your feedback, did that text exist? That's great. Again, we'll change that later an assertion and let's start the live chat. Now, I clicked on the live chat and what do we want to do? We don't want any of those so let's tell us what your issue is. Let's click on that and great, we're now chatting with the Netflix Help Center. Hello, what can we help you with today? Lovely. Let's just send a reply in there, let's say, testing your chat and let's send that in.

So now I'm in an iframe, testing the iframe on the website. I want to chat with an agent, lovely. Can we sign in? No, let's not sign in. Someone will be with you shortly, great. And the wait time now is one minute so we can check that that is there and let's see what happens. So we are now interacting with an iframe. We're able to easily test that iframe, as you can see, and make sure that the chat works as one would expect it to work. So I'm now chatting with Martial, so thank you Martial for replying there to me and I'm just gonna say, hey, I'm just testing the chat. Thanks Martial. And we'll send that in there and then let's not bother the people at Netflix any longer. So let's end that chat. We'll press end, click that button. Yeah, we want to end it and email me a transcript of this chat. That sounds really cool actually.

Let's do that. So click on that and we put in our email. I'm just gonna put Debbie and press send cause I don't want to put my email really. And oh, we've got a, we've sent this transcript to the chat, to the email, so that probably shouldn't have. So we can go and fix that code, right? Based on how we're finding things.

So as you can see, I'm clicking around and I'm even finding bugs as I'm clicking around and writing the test and I'm not even actually writing any code. So cool. Back in the footer, we can open up another domain. I'm not testing another domain. I'm testing this fast.com from the Netflix footer, check that that image is there. Let's click on that button, that show more info button and make sure that the client is there. So again, I'm able to check over multiple domains. I'm able to check iframes and all my code is written for me, as you can see. And I didn't even have to like open any code, right?

Now, obviously we're going to have to change some of this, which we'll go through in a second, but you can see there's my iframe being tested. You can see my other page is being tested and everything here is written for me, bang, right there in VS Code. And I can just go along and make sure that works by literally clicking that button. That's gonna basically run up the browser. It's gonna go through here and you can see it started up the chat and we'll go to that one minute wait and look, Playwright is gonna wait for that to be available. Bang, it's done, it's there. I've got chunia this time. Everything works, going to the next domain, testing that that works and up comes my little button. It's gonna wait for that button. It's waiting, it's waiting, it's not there yet. Once that comes up, bang, I've got there and everything is as expected. So no timeouts had to be set there. We have everything working. So again, we should be using web assertions and you saw I clicked some of those headings. So then I went back over this test and I improved it. So I changed some of them like the logo to be an assertion. So I expect it to be visible and I was able to like look at these page, get by roll heading.

If it's a get by roll heading and I wanna make sure that that is visible. Get by roll link, yes, that's a click event that's perfect. Again, the next one is the URL that's good. Get by page link, click, perfect. And we can go down through the test and every time we see a get by roll heading, that one, we just want it to be an assertion so we can easily then modify each one which makes it a lot easier kind of, you know, to write your test, right? You could totally write this completely all on your own without using code gen if that's what you're into but I just find using code gen a lot easier for getting started.

So let's talk about show reports. So when you've got your tests, this is at the moment now, I need to use the terminal for this. I can't use the VS Code extension. So I wanna run my test first, npx playwright test, and then I wanna show report. npx playwright show report. Now what this gives me is a whole report of my tests and I can see, whoa, it failed on Firefox. It passed on Chromium and WebKit and then I could go in and inspect what happened in Firefox. I can see a whole log of it but I wanna show you the trace because the trace is so much more fun. It's that little icon there, the three little bars. So I can click on that and it will open up the trace or I can go into detail of that and you'll see at the very, very bottom, the trace as well. When I click on the trace, what it will open up and show, well, first of all, let me show you how you can run the traces, right? You can record a trace for each test locally using npx playwright test dash dash trace on. Now normally the trace is run on CI on the first three tries. If your test fails on the first retry a trace is recorded for you. You don't have to do anything to set that up, it's all done. But if you wanted to do it locally, you would do npx playwright test dash dash trace on and then basically open up the report and then click on that and you'll see the trace. So this is a trace here. Basically what the trace is doing is it's recording every single thing that happened during that test. So you'll be able to then go along that timeline and in your own time, go back and forth and kind of really see what's going on. So these are image snapshots of every single action. You can see each one is highlighted with different colors, locator, click frame that was the iframe. So every kind of event is highlighted there and you can basically like go through that, scroll across, go back, kind of gently see what was happening. You can see there my email address, Debbie, you can see I sent, you can see there's no address added in there. You'll be easily able to point out when something went wrong and so that someone is gonna be able to fix it. They can really see, oh look what happened there. Oh I can see, I can really kind of, slow it down and speed up and kind of find out.

Now if you don't want to use the timeline, you can also go to the actions and stuff. Now the actions, what it's gonna do when I click here in the middle. This is a dom snapshot, not an image snapshot, so this is even better. And you can see the click event, the action, the little red dot is highlighted there in every single action. And we'd be able to see like what happened before, what happened after. And we can go to those actions and you can see which one was clicked.

The text thanks for the iframe, we're now in the iframe. What's going on in that iframe and every single one the locator clicked there, they expect it to be visible, the iframe. And then over in the other side, you can see the call, the console, the network, and the source. So you can see at each time during each action, what's going on in that call, the duration, the wait time, which selectors. If it's strict, what value, the whole log waiting for the iframe, the selector is also visible, waiting for the element to be visible, element is visible, enabled and editable. And you saw I actually wrote in there because it is an actual DOM snapshot rather than an image snapshot. So that means you can also open the Chrome DevTools, you can expect the CSS in there, you can really kind of understand what's going on. And you can see then you've got like the different network requests are highlighted there. So one of them we had one network request there, and as you go along, you can see everything that's going on as well in the source code as well. So you can see the source for your test.

Now, this is a PWA, which basically you can then send to someone else so they can then see the source code rather than actually just, you know, having to open up the actual code itself. So the metadata is also there, you see if it's mobile, etc. So this is a really, really good way of debugging your test, I love it also when I'm building, but yeah, it's very, very cool. So make sure you check out the Trace viewer.

Now you can also run your test on CI very easily with GitHub Actions. This is what you're gonna get that's already included for you once you tick that box, which is the GitHub Actions. And once that's happened, you know your test is either gonna fail or they're gonna pass. You're gonna have it red, you're gonna have it green. And if they're red, especially you wanna kind of like find out what's going on. So you can run, look into those actions, go along, check out the log, and you can also see that Playwright Report there. I showed you the report locally, that's where you get the report on CI which you can then download and use Playwright to then serve that up. And you'll be able to click on that and trace and kind of, you know, visualize that trace and see what happened in CI, why did that fail? And you know, debug that and improve your test. So with Playwright testing, you have the VS Code extension. You got Code Gen, my friend, is gonna auto-generate the test for you.

You can run your test in debug mode. You've got locators and frame locators. So you can test iframes. You can see reports, you can open up the Trace viewer and really see a full trace of those you can still do screenshots and videos if you like, but I just love the trace viewer.

So that's by default, but if you prefer videos, totally up to you, you can also do that. Auto-wait built in for all actions. So there's no need to set timeouts. Tests run in parallel for free without you having to do anything which makes it super fast. It runs across all browsers and devices. So emulates mobile devices as well. Intercepts requests, follows all redirects and bypasses cores and manages cookies for you.

So who's using Playwright? Absolutely everybody. I mean, check out that chart. Playwright is open source and free. It's by Microsoft and we absolutely love GitHub stars. So make sure you give us a star. And these are amazing ambassadors. Check them out. They're on our community tab ambassador page and they're creating amazing content on Playwright. So do connect with them and check out their video and courses and blog posts and et cetera and make sure you're following them.

But seriously, I just have one question for you. Are you ready? Are you ready for the question? Are you ready to playwrite? I want to hope to see you all basically in the playwright community creating great tests. Let's improve the testing scene in general. Let's write better code. Let's test that code and let's just have some fun and make testing easy and fun. That's the goal. Thank you very much for your time. Check out our docs. Check out our Twitter. Check out our YouTube and our community Slack channel.

And if you want to reach out for anything, you can find me, Debs underscore Brian on Twitter. My website Debbie dot codes. And I really look forward to seeing so many more of you in the playwright community. Thanks everyone, bye.

And that was such a power packed talk by Debi. So before the talk, we ask the question and now I'd like to invite Debbie on the stage with me and we will discuss the Q and A and the poll questions. Debbie, please join me. Hi. Woohoo. Hi. Thank you Debbie. That was such an amazing talk and actually I learned a lot. So I saw playwright for the first time and I think I'm gonna try and so are many of the audiences, of course, after seeing the talk.

So we asked the question, is like a playwright API can be used with all of these except, and I think 93% of the people say C plus plus. So what do you say? Is that the correct answer? I think they did pretty good. It's quite tricky, right? Because it's C plus, right? And not C plus plus, right? And it's that's if you're not into the C pluses then you might get confused. But yes, playwright works on C sharp or we can say.net, right? Python also, TypeScript, JavaScript and Java. So we have pretty broad like a covers of languages in which you can write playwright tests. Exactly, which is kind of cool, right? Because I mean, if we work in the same company, right? And now our backend developers who are writing in Java for example, or .net, they can continue writing in their language and we're gonna write in TypeScript and you're gonna write in JavaScript because you're afraid of TypeScript, but that's fine. And we'll be able to still use the same testing tool even though we're using different languages and we'll all understand each other's tests but we'll all write them in the way that we understand. So it's kind of cool. That's cool, so like just a question which comes out of my mind after this is that can you write all the different language testing for the same project or like... Yeah, so you can decide to put your tests with your project. So if you're writing a Node.js project, then obviously if your tests are inside that project and you want to run it, you're better off running those in Node, but you can also run them outside of your project or you could just set up a Dev server in a GitHub Actions to kind of like launch that server. So all depends on where your server is running to be able to like run those tests or not, but you can totally test outside your application inside it, whatever works better. Awesome, so there's a question. Mark R asks, yeah, I thought that it would be a question. What are the advantages of using Playwright over Cypress? That's a horrible one. Cause you know, Cypress is great. I love Cypress, you know, I used to work with Cypress.

So what are the advantages over Cypress? I guess what I showed in the talk specifically, be able to test chat applications, go across multiple domains, go into Iframes without anything else, just it just works. Parallel for free and probably other things, but let's not go down that road. But yeah, test your applications. I think that those two are pretty nice frameworks, so yeah.

So we have a question from Murray, with the new API page.getByTestId, does this require the custom attribute to be data test ID or can it automatically detect other custom attributes such as data-test or something similar? Yeah, of course, so we just set one and then if that works for you, perfect. But if not, you could write data my beautiful tests if you want, right? You can change that attribute to whatever you want so you can configure that. Awesome, awesome, yeah, thank you. That sounds a pretty nice idea for attribute name for sure.

Yeah, so yeah, that's also a question for people who might want to get started with PlayWrite. Like how can they get started with PlayWrite? So I recommend you take three minutes out of your day. That's all it's gonna take, three minutes out of your day and literally just go through the getting started guide. I know people go, oh my God docs, I hate docs. I don't want it but it's so simple because literally it just takes you through installing. You can use the VS Code extension, which I showed. And then you don't even have to write a test because you've already been given a test. Press the play button, it's gonna run, right? Now you're on a minute and a half. So now another 30 seconds to get code gen, create a test just by clicking. Bang, you've got the other test run it, and then you can put up on GitHub Actions in another minute, three minutes, run test running up on GitHub Actions up on CI easy. See, three minutes is all it takes, right? To go from zero to hero in playwright, yeah. And if you fail, if you fail, come and talk to me. If it doesn't work for you, come and talk to me. I wanna know why because it should. Awesome, yeah, thank you, thank you.

So also we have a question, in your opinion, what is the future of playwright? The future? The future is that everybody is gonna love testing and they're gonna always be testing their applications. So we never have problems that we click something and it doesn't work, that the user gets frustrated. We're gonna have no frustration in the web anymore because if we make testing easy and fun, more people will test. Testing will just happen and it will become a culture, part of the culture. Like in the Netherlands, cycling a bike, right? That's the culture. Let's get the whole world cycling bikes. Let's get the whole world riding tests.

So yeah, that's what I think the future is. That is a amazing answer. Really, like testing are very common and easy. So everyone gets into it for sure. Thank you.

Also, so what is like, you have been working for playwright for a while, right? So what is your favorite feature of playwright so far? You know, it's kind of hard because I'm between Codegen and the Trace viewer. Codegen I love because I never have to think about which of those locators that you need to choose. I don't think I just click, it tells me which ones and then I can look at it and say, yeah, you know, that's good, looks good, perfect. It writes my tests for me, that saves a lot of time. And then Trace viewer, when the test fails, to be able to do that. But I'm just not really getting a lot of fails tests. So I'm going to say Codegen because it's helping me really get a lot of tests done. But then when I saw it like for me also, it was like that Codegen was like, wow, that sounds super cool. Yeah. Yeah. So, yeah. This is why I have so much time to do sports, right? Because Codegen's writing my tests for me. See now you have secret of Debbie doing three hours and six hours of sport every day. Yeah, so like I already asked you but just for the audience again, what's like how many hours did you do today and what are you going to do next for the sports? Yeah, so I ran to the gym. That was one kilometer. I played a match. That was an hour and a half power match. And I ran home. That was another kilometer. And in an hour's time I'm going to run to the gym for my second match. I've got actually two matches, but they're shorter matches. So I'll be another hour and a half. So I'll be about three hours, 20 minutes. That's just amazing. Like the way you like it really motivates.

So on that note, I'm very, and the other question is how to stay motivated and so energetic. Like how would people be so energetic? Like give us some tips, come on. You literally have to put it in the calendar. It's in my calendar that I need to go to sport. Cause if not, I'm going to sit here and I'm going to code all night long and I'll forget to even have dinner. So literally I put it in, if it's in the calendar, it's going to happen. If it's not in the calendar, it doesn't happen.

Love it. But thank you. Thank you so much Debbie again for this wonderful talk and showing us all about Playwright and answering all the questions. It was really amazing to have you today with us.