Build a Metaframework in 30 Minutes or Less

We've wipecoded our application real quick, or we inherited a legacy large codebase. How do we test it? Is testing easy in 2025? That's a tricky question. And before I jump in, a quick check. Can you please raise your hand if you worked with Playwright before? Great. It's like almost half of the audience, maybe even more.

For the rest of us, a brief introduction to Playwright. Playwright is a very popular end-to-end testing tool. It's very easy to install. With just one command, you get the default browsers, configuration, example test, and even GitHub action. It comes with Code Jam. It allows you to generate tests without writing any code at all.

When tests inevitably fail, Playwright has best-in-class developer experience with built-in debugger, VS Code extension, screenshots, reports, videos, and my favorite, the trace viewer. The trace viewer is a one-stop shop for any test failure. It provides all the test steps along with screenshots, errors, console logs, network requests, and basically everything that we usually check in the browser dev tools. Of course, the mandatory AI slide. Playwright has a couple of nice AI features, copy error as AI prompt, and Playwright MCP server to allow large language models to communicate with Playwright APIs.

Let me give you a quick demo. Here, I'm asking Courser to write a quick conference review draft for my blog. Courser asks Playwright MCP to fetch a conference agenda and Playwright returns an accessibility snapshot of the page, something that is very easy for Courser to digest. As a result, Courser generates a good first draft, even for the talks that were not presented yet. Very cool. Of course, Playwright's main goal is to help us write reliable, predictable tests. And Playwright is a very practical tool. It comes with auto-weighting and web-first assertions. Authentication and multi-user comes out of the box, the same with parallelism and sharding. Finally, it has a great community that provides community plugins, best practices, and integrations with other tools.

And Playwright is a very practical tool. It comes with auto-weighting and web-first assertions. Authentication and multi-user comes out of the box, the same with parallelism and sharding. Finally, it has a great community that provides community plugins, best practices, and integrations with other tools. All in all, Playwright checks all the boxes to make testing easy.

At least, this is what I thought when a year ago, I joined a team at Aetion, a healthcare analytics company that offers a range of products for data scientists in the healthcare sector. It's a highly specialized and regulated business domain that probably has an acronym for pretty much everything in the world. Let me give you an example. In the International Classification of Diseases, that's a real thing, there is a code W6162xd that literally means struck by duck, subsequent encounter. And in such specialized business domain, our team was asked to cover all the major features with automated tests for regulation purposes. The scope included 64 epics of tests.

I still remember this number. For a non-professional like me, the UI looked like gazillion checkboxes, acronyms, tables with random numbers all around, powered by a 5-year-old React application and a 10-year-old Java back-end. Java had its own story going from monolith to microservices, then back, leaving us with a bunch of legacy outdated APIs. But that's for another time. For now, let's focus on React. As the product was evolving, some of the components have grown over a thousand lines of code with those huge use effects, with missing dependencies, dependencies, and all that stuff. React version 17, most of the code untyped. ESLint and Prettier got lost during cloud migration.

Before we move on, I just wanted to mention that this is not an uncommon situation for growing businesses that focus on building new features. Now, having that said, our first reaction to this project can be summarized by a quote from my teammate. This code is impossible to test. Or speaking scientific language, R45.4, irritability and anger. Challenge was not accepted. With all those excuses running in my head, no, it's not my job. Even a junior developer can do that. Even AI can do that. And after five hours of me prompting chat GPT, stupid AI, it will never take my job. Okay. I'll write the tests. At least tell me what kind of tests should I write? Where do I start? We needed a testing strategy.

Now it's time for a brief theoretical blog. Probably the most popular testing strategy out there is the testing pyramid. It promotes breaking software into smaller, testable units. Not our case, to be honest. The other option is to put the pyramid upside down to get the reversed testing pyramid. If I put a bit of ice cream on top, then we get the testing ice cream cone. The default testing strategy for many companies. Not that tasty. Most of the testing is manual and only some of that gets automated.

Another option is the testing diamond and puts main focus on the integration tests. An evolution of the testing diamond that is more popular in the frontend community is the testing trophy by Can'tSeeDots. It clarifies what different types of tests mean in the context of the modern JavaScript ecosystem and also makes static checks such as YesLint and TypeScript more explicit. Not everyone knows, but Kian's favorite strategy is the testing Dorito by Tim Myers. Let me read it for you. From top to bottom. Tests I plan to write. Tests I start writing. Tests I delete because I think they're stupid and they can take a lot of time and they're not worth.

Finally, tests. And the testing Dorito is a good reminder that theory and practice don't always have a perfect match. So without further ado, please welcome brand new testing strategy, the testing mushroom. The cap of the mushroom is all about end-to-end testing. It clears regulation requirements and also helps us build a safety net so that we can start refactoring and introducing integration and unit tests. The testing mushroom is a living organism, so it's expected to change its shape once we learn more.

Great, we've got our strategy, so it's time to execute. The whole team sat down to write as many tests as possible, as fast as possible. And a month later, it was a disaster. It didn't work. The more tests we were writing, the more of them were failing with all the random reasons and it was impossible to figure out what was going on. It was time for an emergency action, namely written documentation. At least the best practices part.

Apparently, we have a misusing playwright API, and here I have a couple of examples for you. The line at the top is the wrong way to check the welcome text on screen, and the line below is the right way to do that. Both work, it's just the first one sometimes randomly fails. And to be honest, the difference is subtle, especially when you review thousands lines of test code. Luckily, there is a simple solution. We installed ESLint plugin Playwright. It fixed all the issues and also helped us learn what was the right way to use playwright. Great.

With that, we continued, and two months into the project, we realized we were behind schedule. So we had to ask our colleagues from other teams for help. This is when we faced our next challenge, test dependencies. As you remember, we are dealing with a quite complex business domain. A typical user journey can take several minutes before you achieve some meaningful result. And for developer, it's tempting to reuse results from previous tests to build new tests on top of them.

Unfortunately, it breaks completely test retries and makes local debugging practically impossible. You never know in which order and which tests to run in order to reproduce the issue. Now, once again, we decided to follow best practices. And one of them says, make tests as isolated as possible. Well, easy to say. We had to enforce it with fully parallel mode and increase the number of workers to three to actually run the tests in parallel. That revealed a few issues that we sorted.

And with that, finally, we were great. Our tests became much more stable and they started catching real bugs like that one at the top right corner. Let's actually see what's wrong with it. Okay, that's my test. I can see immediately what's the issue. It's one of the backend APIs not returning the right response, so I'm pushing it to the backend team. The backend team opens it apparently in the dark mode, and can't understand anything, just like you probably, because it's too small. So they push it back to us. And we push it back to them, and so on and so forth.

Let's look at 63.1, problems in relationship with in-laws. Apparently, our test reports were not readable at all, and we had to improve it. When it comes to making code more readable, we already know how to do that. We break it into smaller logical blocks. In testing, there's a common pattern, page object model, where you organize your code in screens or pages. For example, here I have a login page class, and it contains all actions that you can perform on that screen or page. Login, logout, password reset, etc.

We already had our code organized this way, and apparently, we were just one step away from making our reports as readable. And when I say one step, I literally mean it with step decorator. Step decorator groups lower-level playwright actions into high-level steps, logical steps that you can understand. It doesn't come out of the box, but there's documentation on how you can configure it for your needs. With that, our test reports became much more readable, even for our lovely backend team.

If I zoom out a little bit, this is how the failed test looks in the testing report. It contains all the test steps, screenshots, videos, test retries, tags, history, and all that comes from Allure Reports. Allure Reports is an open-source cross-language, cross-framework tool for building beautiful test reports, so that even non-technical users can figure out what's going on in the test. And as usual, it's very easy to configure it with Playwright. Great! With that, we were on the finish line when our platform engineering team reached out with a much bigger problem. Yes, our tests were green. Yes, they were stable, but they were super slow.

They could block the deployment pipeline for an hour or two, and it was not acceptable. To be honest, we never focused on speed. We had other priorities. As a reminder, we are running tests in full parallel mode in three workers. And with just one configuration change, we went from 51 minutes to 17 minutes. Yes, we just increased the number of workers.

You can see the white space at the beginning. We still have some room for improvement, but it's already a good first step. Of course, it doesn't make each individual test run faster. We just run more of them in parallel. If we compare end-to-end tests to integration tests, integration tests usually run faster because they skip or mock expensive operations.

At Playwright, we have a network API for intercepting HTTP requests, but personally, I'm not a huge fan of this approach. On one project, we mocked all the HTTP requests. Our test was super fast, super green, but they never caught a single bug. Could we gain similar performance improvement, but without actually maintaining mocks? Mocking without mocks, as I call it. This is exactly what Playwright network cache plugin offers.

Could we gain similar performance improvement, but without actually maintaining mocks? Mocking without mocks, as I call it. This is exactly what Playwright network cache plugin offers. Let me show you an example. Imagine we have five slow tests. Each starts with performing some slow operation. Let's say opening a test project. With Playwright network cache, the first test will perform the operation, doesn't matter get or post multiple requests or just one, and will store the result in cache. All the other tests will reuse that cache as mocks. In our case, we actually had 200 of tests like that, and we got back 10 minutes of test running time. That's insane.

Now, if you think it's cheating, you're not wrong, because it's no longer a pure end to end test. That's why in the testing mushroom, our integration tests sit inside the mushroom cup. And I could talk more and more about the testing mushroom, because it's just a lot of fun. But honestly, I made it up completely for the purposes of this talk, to illustrate the approach that we took for this particular project. And your approach will be different. I hope the testing mushroom inspires you to explore the different shapes and strategies out there and find what works for you and makes testing, if not easy, then at least more fun.

With that, let me recap. We wrote all the tests. Mission complete. They're stable and readable. They catch real bugs. They run in full parallel mode. Great job to the team. But there's something else that I didn't mention. An unexpected side effect. Today, one year later, we have React and other major libraries upgraded. We duplicated those outdated integration tests with those legacy mocks that nobody could maintain. The team switched to monorepo to improve developer experience. We migrated to feature-slice design code structure to align better with our business domain and also to start breaking down our code into smaller, testable units. It accelerated TypeScript migration.

Today, building new features is much easier. It's a completely different code base, but I know it all started with testing. So back to the question. Is testing easy? It depends on the right strategy, discipline, and skill. But the good news, everything else becomes easier when you've got your testing right. And my favorite. RS 202.5 obsessed with testing at React Summit 2025. My name is Kate. I'm collaborating on social media. Happy testing. Thank you. We have Slido running. And don't forget that you can add your questions as we discuss. We are going to see who upvotes what.

But why don't we start out with testing approaches? So you went through several different snacks that represent testing approaches. Which one do you think is the best? I think that's an asterisk question. That's a tricky one. Because the whole point of my talk is that it depends on the code base that you handle. And sometimes, like my best is of course the classic testing pyramid. But it's not always possible to start with it if you deal with legacy code base or, you know, the situation is what it is. So I would say find what works for you. And then look around because there are some smart people that come up with some interesting ideas. Absolutely.

But I think we can all agree what is the cutest testing strategy, which is obviously the mushroom. I don't know about the Dorito. Maybe. It depends on the flavor. Okay. So let's see. How about the parallelism? We got a couple of questions about that.

So is there a way to disable parallelization for certain test suites that have long initial load times? Yes. I think what the person wants to achieve is perfectly possible. Playwright is a very configurable tool. But I would change that. And we forced ourselves in this situation where we had to deal with full parallel mode and it helped us a lot to actually follow best practices and help us, you know, maintain it as it goes. Awesome. Awesome.

And on that same subject. So in the parallel workers, like how, I guess, you know, how many did you finally use? Or perhaps the wider question is how do you determine how many to use? And were you running in CI in continuous integration? Yeah, yeah. Of course. We currently use 12 workers in CI. We don't use it on local, for example. Local, I use four workers maximum. It depends. In our case, we just picked the largest machine that was available for us in CI and that's it. It was simple. The number of CPUs. Yeah. So that's basically what you got. Right? Awesome.

And so how about those end to end tests that are... How do you ensure that they aren't flaky? Because you have some issue with the server, some random downtime, something else goes wrong. What do you do against flaky end to end tests? That's a great question. So if you follow playwright best practices, then you reduce, you know, unnecessary flakiness that comes from the fact that you're using browser and there are some latencies, et cetera. Now if your server is down, there's nothing that can help you. And that's at the core of the end to end test as opposed to just the whole thing. So we let them fail if the server fails and we ping our lovely backend team to fix that. Now there are mixed up approaches, as I mentioned, with this same integration tests, when you mock some of those things, this is the other approach that you can use and you can go creative. Yeah. Yeah.

Okay. Makes sense. So again, got to roll with the punches. Right? You got to get creative for whatever the problem is at hand.

Okay. We have a question about a very specific question. What is the difference between at step and test dot step? That's the same thing. At step is a decorator that wraps the methods for you. Beautiful. So you don't need to modify all your code base or your existing test, you just add the decorator. Yeah. And that's maybe a personal preference thing again, like what does your team prefer? What's your style? Are you used to working with decorators? Are you not? Cool.

So let's see. So many great new questions are coming in. I love to see this, even though we're having a bit of a silent disco and so it, you know, it's very quiet in real life here. We're all having an awesome virtual interaction, which is bringing me back to like five years ago. Anybody? 2020 vibes? Okay. So, all right. How about, how do you approach, for example, dependencies on user registration without repeating registrations every time? Yeah, that's a great question. First of all, we have pre-configured test user like seed data that we have and they already contain some test users. So we test registration only once, but then we use those pre-configured users. Awesome.

So as the questions are coming in, don't forget, you can upvote the ones that you want to hear answered, but if your question isn't answered, or if you were perhaps too enraptured by the joys of thinking about testing strategies to ask it here online on Slido, those of you that are in-person will have a chance to join Kate's Q&A corner later on. All righty. But in the meantime, back to our virtual disco, silent disco party. I wish we had like roller skates or something, no? Just me? I'm sure nobody would get hurt. That would be fine.

Speaking of logins, I guess, and registration, so any thoughts on optimizing login generators or util functions that will be run for each test? So this person has had trouble with, or their team has had trouble with parallelization and timeouts when hitting various endpoints? Okay. The answer is quite long, so let's chat after the talk. In short, for repeated code, we use fixtures a lot. They mean different things in Playwright than other things. And for stability, we need to chat because it depends. It depends, right? It depends. The software engineer's most used sword. Absolutely.

I think we have time for a couple of more questions, if you're up for it. I'm up for Thunderstalk, honestly. So let's wrap it up. There are going to be more talks on the alternate stage. So yes. But don't worry, if you do miss any of the talks because you're, again, having such a great time silently talking about testing up here, don't worry, they'll record it. You can catch them later. So we will make sure to catch that. And once you've had a chance to cool down as well. But let's see, would you be willing to do one more for our audience here? Can we get a virtual woo to encourage this? Absolutely amazing. Fabulous. All right, let's see. Our most popular question at the end is, are you using Playwright sharding in CI and CD runs? And if not, why not? We just don't need it. We use a large machine. As soon as we need it, we will use it because sharding requires a bit of massaging of the reports.

We use a large machine. As soon as we need it, we will use it because sharding requires a bit of massaging of the reports. So you need to merge them back. And we don't want to do that. Can we grow big enough? We will do that for sure. So again, depending on the need, right? Like depending on what your actual capacity is and what your actual... But it's a great question. Yeah. Awesome. Awesome.

Well, I love that we have such a colorful discussion going on about what I'm now going to think of as a very hunger-inducing part of software development, testing strategy. So can we all please give a virtual and real huge, huge thank you to the amazing Kate and all of the testing shapes. Thank you very much. Thank you so much, Kate.