At the Top of the Pyramid: Playwright Testing at Scale

We've wipe-coded our application real quick, or we inherited a legacy large codebase. How do we test it? Is testing easy in 2025? That's a tricky question. And before I jump in, a quick check. Can you please raise your hand if you worked with Playwright before? Great. It's like almost half of the audience, maybe even more.

For the rest of us, a brief introduction to Playwright. Playwright is a very popular end-to-end testing tool. It's very easy to install. With just one command, you get the default browsers, configuration, example test, and even GitHub Action. It comes with Code Jam. It allows you to generate tests without writing any code at all. You basically click around in your browser, on your website, and you get back a simple, basic, but functioning test code.

When you have more advanced test scenarios, Playwright has all the features. Visual comparisons, snapshots, API testing, Chrome extensions testing, component testing. If for some reason you don't like TypeScript, there are other options as well. When tests inevitably fail, Playwright has best-in-class developer experience. It has a built-in debugger, VS Code extension, screenshots, reports, videos, and my favorite, the Trace Viewer.

The Trace Viewer is a one-stop shop for any test failure. It provides all the test steps along with screenshots, errors, console logs, network requests, and basically everything that we usually check in the browser dev tools. Of course, the mandatory AI slide. Playwright has a couple of nice AI features, copy error as AI prompt and Playwright MCP server to allow large language models communicate with Playwright APIs. Let me give you a quick demo. Here, I'm asking Courser to write a quick conference review draft for my blog. Courser asks Playwright MCP to fetch a conference agenda, and Playwright returns an accessibility snapshot of the page, something that is very easy for Courser to digest.

And as a result, Courser generates a good first draft, even for the talks that were not presented yet. Pretty cool, but of course, Playwright's main goal is to help us write reliable, predictable tests, and Playwright is a very practical tool. It comes with auto-weighting and web-first assertions. Authentication and multi-user comes out of the box, the same with parallelism and sharding. Finally, it has great community that provides community plugins, best practices, and integrations with other tools. All in all, Playwright checks all the boxes to make testing easy.

At least this is what I thought when a year ago, I joined a team at ATO, a healthcare analytics company that offers a range of products for data scientists in the healthcare sector. It's a highly specialized and regulated business domain that probably has an acronym for pretty much everything in the world. Let me give you an example. In the International Classification of Diseases, that's a real thing, there is a code, W6162XD, that means, it literally means, struck by a duck, subsequent encounter. And in such a specialized business domain, our team was asked to cover all the major features with automated tests for regulation purposes.

The scope included 64 epics of tests. I still remember this number. For a non-professional like me, the UI looked like gazillion checkboxes, acronyms, tables with random numbers all around, powered by five-year-old React application and ten-year-old Java backend. Java had its own story going from monolith to microservices, then back, leaving us with a bunch of legacy outdated APIs, but that's for another time. For now, let's focus on React. As the product was evolving, some of the components have grown over a thousand lines of code with those huge use effects, with missing dependencies, and dependency array, and all that stuff.

React version 17, most of the code untyped. ESLint and Prettier got lost during cloud migration. And before we move on, I just wanted to mention that this is not an uncommon situation for growing businesses that focus on building new features. Now, having that said, our first reaction to this project can be summarized by a quote from my teammate. This code is impossible to test. Or, speaking scientific language, R45.4, irritability and anger. Challenge was not accepted. With all those excuses running in my head, no, it's not my job, even a junior developer can do that, even AI can do that, and after five hours of me prompting chatgpt, stupid AI, it will never take my job, okay, I'll write the test, at least tell me what kind of test should I write, where do I start? We needed a testing strategy.

Now it's time for a brief theoretical block. Probably the most popular testing strategy out there is the testing pyramid. It promotes breaking software into smaller testable units. It's not our case, to be honest.

The other option is to put the pyramid upside down to get the reverse testing pyramid. If we put a bit of ice cream on top, then we get the testing ice cream cone, the default testing strategy for many companies. Not that tasty. Most of the testing is manual, and only some of that gets automated.

Another option is the testing diamond, and puts main focus on the integration tests. Another evolution of the testing diamond that is more popular in the front-end community is the testing trophy by Kent C. Dots. It clarifies what different types of tests mean in the context of the modern JavaScript ecosystem and also makes static checks such as slint and TypeScript more explicit.

Not everyone knows, but Kent's favorite strategy is the testing Dorito by Tim Meyers. Let me read it for you from top to bottom. Tests I plan to write. Tests I start writing. Tests I delete because I think they're stupid, and they can take a lot of time and they are not worth. Finally, tests! And the testing Dorito is a good reminder that theory and practice don't always have a perfect match.

So, without further ado, please welcome brand-new testing strategy, the testing mushroom. The cup of the mushroom is all about end-to-end testing. It clears regulation requirements and also helps us build a safety net so that we can start refactoring and introducing integration and unit tests. The testing mushroom is a living organism, so it's expected to change its shape once we learn more.

Great, we've got our strategy, so it's time to execute. The whole team sat down to write as many tests as possible, as fast as possible. And a month later... It was a disaster. It didn't work. The more tests we were writing, the more of them were failing, with all the random reasons and it was impossible to figure out what was going on. It was time for an emergency action. Namely, written documentation.

At least the best practices part. Apparently, we were misusing the Playwright API. And here I have a couple of examples for you. The line at the top is the wrong way to check the welcome text on screen. And the line below is the right way to do that. Both work, it's just the first one sometimes randomly fails. And to be honest, the difference is subtle. Especially when you review thousands of lines of test code.

Luckily, there is a simple solution. We installed ESLint plugin Playwright. It fixed all the issues and also helped us learn what was the right way to use Playwright. Great, with that, we continued. And two months into the project, we realized we were behind schedule. So we had to ask our colleagues from other teams for help. This is when we faced our next challenge.

Test dependencies. As you remember, we are dealing with a quite complex business domain. A typical user journey can take several minutes before you achieve some meaningful result. And for a developer, it's tempting to reuse results from previous tests to build new tests on top of that. Unfortunately, it breaks completely test retries and makes local debugging practically impossible. You never know in which order and which tests to run in order to reproduce the issue.

Now, once again, we decided to follow best practices. And one of them says, make tests as isolated as possible. Well, easy to say, we had to enforce it with fully parallel mode and increase the number of workers to three to actually run the tests in parallel. That revealed a few issues that we sorted. And with that, finally, we agreed. Our tests became much more stable, and they started catching real bugs, like that one in the top right corner.

Let's actually see what's wrong with it. Okay, that's my test. I can see immediately what's the issue. It's one of the backend APIs not returning the right response. So I'm pushing it to the backend team. The backend team opens it, apparently in the dark mode, and can't understand anything, just like you probably, because it's too small. So they push it back to us, and we push it back to them, and so on and so forth.

If I zoom out a little bit, this is how the failed test looks in the testing report. It contains all the test steps, screenshots, videos, test retries, tags, history, and all that comes from Allure Reports. Allure Reports is an open-source cross-language, cross-framework tool for building beautiful test reports so that even non-technical users can figure out what's going on in the test. And as usual, it's very easy to configure it with Playwright.

In testing, there's a common pattern, page object model, where you organize your code in screens or pages. For example, here I have a login page class, and it contains all actions that you can perform on that screen or page. Login, logout, password reset, et cetera. We already had our code organized this way. And apparently, we were just one step away from making our reports as readable.

And when I say one step, I literally mean it with StepDecorator. StepDecorator groups lower-level Playwright actions into higher-level steps, logical steps that you can understand. It doesn't come out of the box, but there's documentation, how you can configure it for your needs. With that, our test reports became much more readable, even for our lovely backend team.

Great, with that, we're on the finish line when our platform engineering team reached out with a much bigger problem. Yes, our tests were green. Yes, they were stable, but they were super slow. They could block the deployment pipeline for an hour or two, and it was not acceptable. To be honest, we never focused on speed. We had other priorities.

Yes, they were stable, but they were super slow. They could block the deployment pipeline for an hour or two, and it was not acceptable. To be honest, we never focused on speed. So as a reminder, we're running tests in full parallel mode and free workers. With just one configuration change, we went from 51 minutes to 17 minutes. We increased the number of workers, allowing more tests to run in parallel. End-to-end tests compared to integration tests usually run faster due to skipping or mocking expensive operations.

In Playwright, there's a network API for intercepting HTTP requests, but some find working without mocks more efficient. The Playwright network cache plugin offers a way to gain performance improvement without maintaining mocks. By caching responses, subsequent tests can reuse the stored results, saving time. This approach may not be a pure end-to-end test, but it significantly optimizes test running time. The testing mushroom analogy is used to illustrate the testing strategy employed, emphasizing efficiency over traditional methods.

In Playwright, we have a network API for intercepting HTTP requests. But personally, I'm not a huge fan of this approach. On one project, we mocked all the HTTP requests. Our tests were super fast, super green, but they never caught a single bug. Instead, we were constantly regenerating and updating the mocks. Not super cool. Could we gain similar performance improvement, but without actually maintaining mocks? Working without mocks, as I call it.

This is exactly what Playwright network cache plugin offers. Let me show you an example. Imagine we have five slow tests. Each starts with performing some slow operation, let's say opening a test project. With Playwright network cache, the first test will perform the operation, doesn't matter if you get or post multiple requests, or just one, and will store the result in cache. All the other tests will reuse that cache as mocks. In our case, we actually had 200 tests like that and we got back 10 minutes of test running time. That's insane.

Now, if you think it's cheating, you're not wrong, because it's no longer a pure end-to-end test. That's why in the testing mushroom, our integration tests sit inside the mushroom cup. And I could talk more and more about the testing mushroom, because it's just a lot of fun, but honestly, I made it up completely for the purposes of this talk, to illustrate the approach that we took for this particular project. And your approach will be different. I hope the testing mushroom inspires you to explore the different shapes and strategies out there and find what works for you and makes testing, if not easier, then at least more fun.

With that, let me recap. We wrote all the tests. Mission complete. They are stable and readable. They catch real bugs. They run in full parallel mode. Great job to the team.

But there is something else that I didn't mention, an unexpected side effect. Today, one year later, we have React and other major libraries upgraded. We duplicated those outdated integration tests with those legacy mocks that nobody could maintain. The team switched to Monorepo to improve developer experience.

We migrated to feature-sliced design code structure to align better with our business domain and also to start breaking down our code into smaller, testable units. It accelerated TypeScript migration. Today, building new features is much easier. It's a completely different code base, but I know it all started with testing. So back to the question, is testing easy? It depends on the right strategy, discipline, and skill. But the good news, everything else becomes easier when you've got your testing right.

Here's my favorite. RS 202.5, Obsessed with Testing, at React Summit 2025. My name is Kate. I'm collaborating on social media. Happy testing. Thank you.

We have Slido running. And don't forget that you can add your questions as we discuss. And yeah, we're going to see who upvotes what.

But why don't we start out with testing approaches? So you went through several different snacks that represent testing approaches. Which one do you think is best? I think that's asterisk question. Yeah, that's a tricky one because the whole point of my talk is that it depends on the code base that you handle. And sometimes, like my best is, of course, the classic testing pyramid, but it's not always possible to start with it if you deal with a legacy code base or, you know, the situation is what it is.

So I would say find what works for you and then look around because there are some smart people that come up with some interesting ideas. Absolutely. But I think we can all agree what is the cutest testing strategy, which is obviously the Dorito. I don't know about the Dorito. Maybe. Depends on the flavor.

OK, so let's see. How about the parallelism? We got a couple of questions about that. So is there a way to disable parallelization for certain test suites that have long initial load times? Yes, I think what the person wants to achieve is perfectly possible. Playwright is a very configurable tool, but I would change that. Change that. So we forced ourselves in this situation where we had to deal with full parallel mode and it helped us a lot to actually follow best practices and help us maintain it as it grows. Awesome. Awesome. And on that same subject, so in the parallel workers, how many did you finally use? Or perhaps the wider question is, how do you determine how many to use? And were you running in CI? In continuous integration? Yeah. We currently use 12 workers in CI. We don't use it on local, for example.

Local, I use four workers maximum. In our case, we just picked the largest machine that was available for us in CI and that's it. It was simple. The number of CPUs. Yeah. Work with what you got, right? Awesome.

And so how about those end-to-end tests that are – how do you ensure that they aren't flaky? Because you have some issue with the server, some random downtime, something else goes wrong. What do you do against flaky end-to-end tests? That's a great question. So if you follow Playwright best practices, then you reduce unnecessary flakiness that comes from the fact that you're using browser and there are some latencies, et cetera. Now if your server is down, there's nothing that can help you. And that's at the core of the end-to-end test as opposed to just the whole thing.

So we let them fail if the server fails and we ping our lovely backend team to fix that. Now there are these mixed approaches, as I mentioned, with this same integration tests when you mock some of these things. This is the other approach that you can use. And you can go creative. Yeah. Yeah. Okay. Again, you got to roll with the punches, right? You got to get creative for whatever the problem is at hand. Okay. We have a question about, a very specific question. What is the difference between at step and test.step? That's the same thing. It's just at step is a decorator that wraps the methods for you. Beautiful. So you don't need to modify all your code base or the existing test, you just add the decorator. Yeah. So that's the performance thing. Again, what does your team prefer? What's your style? Are you used to working with decorators? Are you not? Cool.

So, let's see. So many great new questions are coming in. I love to see this. Even though we're having a bit of a silent disco and so it's very quiet in real life here, we're all having an awesome virtual interaction, which is bringing me back to like five years ago. Anybody? 2020 vibes? Okay.

How do you approach, for example, dependencies on user registration without repeating registrations every time? Yeah. That's a great question. First of all, we have pre-configured test user like seed data that we have and they already contain some test users. So we test registration only once, but then we use those pre-configured users. Awesome.

As the questions are coming in, don't forget you can upvote the ones that you want to hear answered. But if your question isn't answered, or if you were perhaps too enraptured by the joys of thinking about testing strategies to ask it here online on Slido, those of you that are in person will have a chance to join Kate's Q&A corner later on. All righty.

All righty. But in the meantime, back to our virtual disco, silent disco party. I wish we had like roller skates or something. No? Just me? I'm sure nobody would get hurt. That would be fine.

Okay. So, speaking of logins, I guess, and registration, so any thoughts on optimizing login generators or util functions that will be run for each test? So this person has had trouble with, or their team has had trouble with parallelization and timeouts when hitting various endpoints. The answer is quite long, so let's chat after the talk. In short, for repeated code, we use fixtures a lot. They mean different things in playwright than in other things.

And for stability, we need to chat, because it depends. It depends. It depends. The software engineer's most used sword. Absolutely. I think we have time for a couple of more questions, if you're up for it. I'm up for a talk, honestly. Let's rock and roll. There are going to be more talks on the alternate stage, so yes. But don't worry if you do miss any of the talks, because you're, again, having such a great time silently talking about testing up here, don't worry. They'll record it. You can catch them later.

They'll record it. You can catch them later. So we will make sure to catch that. Once you've had a chance to cool down as well. But let's see, would you be willing to do one more for our audience here? Can we get a virtual woo to encourage this? Absolutely. Amazing. Fabulous.

Let's see. Our most popular question at the end is, are you using playwright sharding in CI and CD runs? And if not, why not? We just don't need it. We use a large machine. As soon as we need it, we will use it. Sharding requires a bit of massaging of the reports, so you need to merge them back. And we don't want to do that. When we grow big enough, we will do that for sure.

So again, depending on the need, right? Depending on what your actual capacity is and what your actual needs are. But it's a great question. Awesome. Awesome. Well, I love that we have such a colorful discussion going on about what I'm now going to think of as a very hunger-inducing part of software development, testing strategy. So can we all please give a virtual and real huge, huge thank you to the amazing Kate and all of the testing shapes. Thank you very much. Thank you so much, Kate. Thank you.