Your Tests Lack Vision: Adding Eyes to your Automation Framework

This ad is not shown to multipass and full ticket holders
JSNation US
JSNation US 2025
November 17 - 20, 2025
New York, US & Online
See JS stars in the US biggest planetarium
Learn More
In partnership with Focus Reactive
Upcoming event
JSNation US 2025
JSNation US 2025
November 17 - 20, 2025. New York, US & Online
Learn more
Bookmark
Rate this content

In this talk, you’ll learn how visual validation works, see a live integration into an existing test code base, and discuss the pros and cons of using various visual validation techniques.

This talk has been presented at TestJS Summit - January, 2021, check out the latest edition of this JavaScript Conference.

FAQ

Unintentional blindness in software testing occurs when something new and unexpected enters your view, and you miss it because you're focused on another feature or aspect. This often happens when testing software, leading to missed issues in the application.

Automating tests can script in unintentional blindness by focusing only on pre-defined steps and checks, thereby missing unexpected issues such as visual bugs or usability problems that could affect the user experience.

Visual bugs are issues in the application's look and feel, such as text being the same color as the background, elements bleeding off the page, or overlapping elements. These bugs are important to detect because they can significantly impact user experience and trust in the application.

Yes, examples include Cineworld's misaligned labels and buttons, OpenTable's confusing modal with misaligned elements, Amazon's visual bug when increasing item quantity, Facebook Marketplace's overlapping text, and Instagram's jumbled sponsored content. These issues can lead to loss of trust and business.

Visual testing involves taking a screenshot of the application when it looks correct and comparing it against subsequent screenshots during regression tests. If there are any differences, the test fails, helping to catch visual bugs and eliminate unintentional blindness.

Pixel-to-pixel comparison is not ideal because it is extremely flaky and can fail due to minor, insignificant changes like cursor blinks or hover effects. This can block deployments and require unnecessary investigation.

Applitools improves visual testing by using machine learning to mimic human eye and brain, highlighting and detecting only significant differences in the application. This reduces false positives and makes visual testing more reliable.

Yes, visual testing can be used for pages with dynamic content by using machine learning algorithms like layout matching, which detect patterns rather than pixel-to-pixel changes. This is useful for content that changes frequently, like news sites or user lists.

The Applitools UltraFast Grid allows you to run visual tests across multiple browsers and devices simultaneously by capturing the state of your application and blasting it across all devices. This speeds up testing and ensures cross-platform consistency without needing physical devices.

Yes, Applitools offers accessibility testing capabilities, including a contrast tester that ensures all elements adhere to accessibility guidelines like AA standards, helping to make applications more accessible to all users.

Angie Jones
Angie Jones
28 min
15 Jun, 2021

Comments

Sign in or register to post your comment.
Video Summary and Transcription
Today's talk is about the importance of visual testing in software development. Visual bugs can easily be missed when relying solely on automated tests. These bugs can have a significant impact on user experience and can even cost businesses customers. Applitools offers a machine learning approach to highlight relevant differences in visual testing. Their eyes API and Cypress integration make it easy to add visual testing to existing tests. With Applitools' ultra-fast grid, visual testing can be done across multiple platforms and devices simultaneously.

1. Introduction to Visual Testing

Short description:

Today's talk is about visual testing. We all suffer from what's called inintentional blindness when testing software. Automating tests can help ensure that steps and verifications are recorded, but it can also lead to missing unexpected issues like the look and feel of the application.

Hello, everyone. Today's talk is about visual testing. But first let's start off with a little game of whodunnit.

Clearly somebody in this room murdered Lord Smythe, who at precisely 3.34 this afternoon was brutally bludgeoned to death with a blunt instrument. I want each of you to tell me your whereabouts at precisely the time that this dastardly deed took place.

I was polishing the brass in the master bedroom. I was buttering His Lordship's scones below stairs, sir. What, I was planting my petunias in the potting shed. Constable, arrest Lady Smythe. But how did you know? Madam, as any horticulturist will tell you, one does not plant petunias until May is out. Take her away. That's right, madam. It's just a matter of observation. The real question is how observant were you.

Clearly somebody in this room murdered Lord Smythe. Who, at precisely 3.34 this afternoon, was brutally bludgeoned to death with a blunt instrument. I want each of you to tell me your whereabouts at precisely the time that this dastardly deed took place. I was polishing the brass. In the master bedroom. I was buttering his lordship's scones below stairs and so. But I was planting my petunias in the potting shed. Constable, arrest Lady Smythe. Did you all happen to see any of the differences? Maybe the biggest one where the dead man got up and walked away? Probably not. And that's because we all suffer from what's called inintentional blindness. Inintentional blindness is where something new and unexpected enters your view, and you miss it because you're so focused on something else. And this is especially true when we talk about testing our software. We're focused on whatever feature it is that we've written and that's the one that we want to work, and we're missing everything that's going on around it. So how do we typically compensate for this? We automate the test. Because if we automate the test, we know that all of our steps will be recorded, and all of our verifications will be verified, and we can guarantee that those things will happen every single time. But what about the unexpected things, right? Sorry to tell you, but when you automate your test, you're actually scripting in the unintentional blindness, and you miss other things that may be happening in your application, such as the look and feel, right? Using the traditional automation tools, we automate those tests.

2. Visual Bugs and Examples

Short description:

Visual bugs can be easily missed when relying solely on automated tests that check for the presence of elements in the DOM. These bugs can include text that is unreadable due to color blending, overflowing elements, or elements covered by others. Even large companies like Cineworld and OpenTable have experienced visual bugs that can erode user trust in their applications.

It's looking in the DOM, and that's where it's interrogating to give us the answer to our questions. For example, is this text present on this screen? It looks in the DOM. The text is present. It tells us, yes, the test passes. But what about if that text is the same color as the background and our users can't see it? What if it's bleeding off the edge of the page, right? What if it's covered by some other element, and therefore our users can't read it, or interact with it, if it's a button or something?

These are visual bugs. And when I talk about visual bugs and visual testing to solve for visual bugs, lots of times people will say, oh, that's interesting. You know, and think of it as this nice-to-have, but let me tell you, visual bugs live in everyone's applications, big or small. Let me give you a couple of examples. Here is Cineworld. Cineworld is the second largest cinema chain on the planet. And they had a visual bug on the page that asks you if you want to store your credit card information. This looks a little funny. Ha ha, we can laugh about it, right? These labels aren't aligned with the buttons, but let me ask you another question. Would you store your credit card information on this page? I wouldn't. Why not? Well, just as Kim has alluded in this tweet, it looks like they haven't tested their application. And if you look at this, you think, oh boy, if they miss this, who knows what else they miss, right, on the back end with my credit card. So no, of course not, I'm not gonna store my credit card information on this screen. But I have a theory. I think they did test this. In fact, I think they automated the test for this. If they automated the test for this, then with querying the DOM, these labels are there, the radio buttons are there. It's not a problem to interact with them. But from a user perspective, this is really bad, right? You lose trust in the application.

Here's another one, OpenTable. OpenTable is used to make restaurant reservations. I used to live in North Carolina a couple of years ago and I went back to visit and I wanted to catch up with some of my friends. I went to make a nice size reservation at one of my favorite barbecue places. North Carolina's known for barbecue. The barbecue place was The Pit and I was making a reservation for seven people. They gave me some times to choose from and I selected the 7 pm slot.

3. Visual Bug Impact on User Experience

Short description:

When I selected this slot, a modal appeared and it took me a minute to figure out what am I supposed to do here because there's nothing in the center. I noticed a couple of select buttons in the corner, but they were not aligned and there was no indication of what I was selecting. I had to open Chrome DevTools to find out that the labels were there, just not positioned correctly. This visual bug could have cost the business a lot of customers.

When I selected this slot, a modal appeared and it took me a minute to figure out, what am I supposed to do here because there's nothing in the center, right? As I got to looking around a bit, I noticed that over here in this corner, I can see a couple of select buttons, but this made me even more confused. What am I selecting? Why are there two of them that are not aligned and there's nothing here telling me what I'm selecting? So I did what any of you would do. I opened Chrome DevTools and I began to dig around the Dom to try to figure out what was going on. And when I did that, I realized that the labels actually were there. They were just as far away from the buttons as possible. So me being who I am, I did complete my reservation and I got to have barbecue, but I couldn't help but think about the many, many customers that are not engineers that have no idea what Chrome DevTools are. They're not looking in the dom to make a reservation on your application. No, they're closing this out. And if they're a millennial like me, they surely aren't gonna give you a phone call, they move on to another restaurant, right? So think about how much business was lost because of this visual bug.

4. Visual Bugs and Validation

Short description:

These visual bugs can occur even in your favorite tech companies like Amazon, Facebook, Instagram, Twitter, and even Google. Visual validation can be automated through visual testing, where screenshots are taken and compared for any differences in subsequent runs. However, the traditional pixel-to-pixel comparison technique used by many vendors is not ideal. It is prone to flakiness and may not catch subtle visual bugs.

These are what else. This happens to your favorite tech companies as well. Like Amazon, on their mobile view, someone not only wanted to buy something, they wanted to increase the quantity, meaning multiply the amount of money that they were wanting to spend in their face with this visual bug. This is Facebook marketplace specifically. So you go to the marketplace, you're trying to shop for something, look at the text over here and how it's just kind of going down the side of the page and overlapping on the images. This is very hard to read. This doesn't make me wanna spend money, right? But if I queried the Dom and asked if that text was present, it'd say, yep, it sure is. This is how this kind of stuff makes it to production.

Instagram, this is one of my favorite ones because this is sponsored content. Meaning someone paid Instagram for premium placement of their advertisement and this is what they got instead. All of the text is jumbled on top of each other, right? Again, querying the Dom, is this text present? Is this text present? Is this text present? Yes, yes, yes, ship it, right? Twitter, same thing. Overlapping text, automated test wouldn't be able to catch that. Google, even Google, my goodness. This one is not as bad as the others, but hey, it is a point of purchase. And so some highly paid Google engineer has to stop innovating new features to go and fix this. Fortunately, we don't have to live like this, people.

Visual validation can be automated. So what is visual validation exactly, or visual testing? Visual testing is where when you write your very first test for a feature, you know, the first time, and you verify that yes, this looks exactly how I want it to look. At that moment, we can take a screenshot of your application. And then for every regression run, we run another screenshot, take another one and compare the two. If there are any differences, your test fails. So this catches everything and it eliminates the unintentional blindness, right? But be careful. Visual testing is not something that's new. In fact, it's been around in the industry for a while, but the technique that many vendors were using was not ideal for testing, right? So the way that most of the visual testing solutions will compare the images is pixel to pixel. Well, that's extremely flaky, let me show you. So here is Google and let's say that very first screenshot is when our test is perfect, our application is perfect. And so that's the baseline screenshot. In the second screenshot, that's a regression run, we compare it in the regression test fields. Who can see why? Yep, that Google search button is bolded in the second one. Well, when your cursor is over that button, that's the hover effect.

5. Automated Testing and Visual Testing

Short description:

Automated tests can be blocked by unexpected elements like cursors, leading to deployment issues. Applitools offers a machine learning approach to highlight and detect only what matters in tests. By comparing pixel-to-pixel and Applitools machine learning results, the latter highlights relevant differences. Adding visual testing to existing tests is simple with Applitools' eyes API and Cypress integration.

So this is an automated test, maybe running as part of CI or something like that. You really didn't have control over where the cursor was happened to have landed and therefore now your deployment is blocked, not what you want.

Here's another one, who can see why this one failed? Yep, because of that cursor behind the word bad. Cursors blink. So if it happens to take the screenshot when the cursor is solid and then happens to take a regression screenshot when it's not, that can cause a failure as well.

You don't want your deployments blocked for this and having to investigate why that happened, nor do you wanna write automation code that tries to stop this sort of thing, right? Fortunately, Applitools has come out with a better approach to doing this using machine learning. So it mimics the human eye and brain and is able to highlight and detect only what matters in the test.

So I have another game here. You all don't get to play because you let the dead man walk in the first game. But in this one, I ran this through pixel to pixel and I ran it through Applitools machine learning algorithm to just see like, what's the difference and how does it detect. And this is the one with the pixel to pixel, notice everything is highlighted here, like every little white shift change, it's not met for testing. Versus when I ran it through the Applitools algorithm, it highlighted the things that I would have highlighted as a human being, right?

So okay, let's add this to a test, an existing test. So here's our application in the test, single page app, very simple, eight of my favorite books on test automation. You enter some text into that field and that will filter down the books, so the ones that have tests in the title, all right? So here's the test to do this. This is written in Cypress with JavaScript, but Applitools has SDKs for all of the different automation tools and all of the different programming languages. So if you use something different, good for you, but this is JSConf, right?

So, okay, so this is Cypress JavaScript. We are querying the word test. We are expecting these five books to come back. We enter that into the field, and then we have two validations here. One validates that the number of books that are visible is the same as what we expect, and each of those titles that's in the expected books array actually is visible, all right?

Great, so now I get a little wonky with the CSS and I want to add a bouncy effect, but I don't really know what I'm doing, but that's okay because I have test, which means I can do whatever I want. So I add line six here to this CSS and I check it in, right? My test will let me know if anything's wrong. Again, here's my test, look at it carefully. And this is what the application looks like with that new CSS. Will my test catch this? Let's see. I run this with Cypress again, notice all the green here. There's no red, even though I have literally flipped my application upside down. That's problem. That's why we need visual testing.

So how do we add it? Very simple. Three lines of code, almost like poetry, right? Starting on line three, we use the Applitude's eyes API and it integrates so nicely with Cypress.

6. Visual Testing Process

Short description:

So look here, becomes a side command. We say, open your eyes. The magic happens when we check the window and take a screenshot. Applitude stores the images on their cloud. When running the test, it detects differences and highlights them. We can scope the test down to specific regions using CSS selectors. However, we need to be careful not to reintroduce unintentional blindness. We can mix visual and functional assertions to make our tests more powerful.

So look here, becomes a side command. We say, open your eyes. We give it the name of our application. The next line, this is the magic, check the window, this takes the screenshot. It sends it out to the Applitude's cloud. Applitude stores all of your images on their cloud so you don't have to worry about disk space or anything like that. Final line, close your eyes. We're done here, folks.

So we run this test and now, notice the test does fail because it detects, girl, you flipped your books upside down, right? And it highlights all of the differences there for me, okay? So this is great for removing unintentional blindness, right, but we could even scope this down. What if we only wanted one book? For example, look at line two. I'm just gonna search for agile testing, knowing that there's only one book that comes back. And so I don't wanna test the entire page because there could be a lot going on in this page that I don't really care about and I wanna scope it down to the book. To scope it down, I use that same check window command, but I pass in a target, which saying, I wanna target a specific region and I give it the CSS selector for that region, okay? So that's the ID. And then notice the screenshot will only capture this one book, but when we scope it down like this, we have to be very careful because we could reintroduce the inintentional blindness. For example, let's say that this filter is broken. I enter the word test here and none of the books go away. Would our visual test pass or fail? If you said that it would pass, you are correct. Because remember, we only had a picture of that one book. We said, hey, look for that one book. Is it visually perfect? It sure is. It passes. So we have to still use our brain. Even though we're using machine learning in our testing, we are still in control. So I can mix and match my assertions. It doesn't have to be all visual. I can add that functional one back in, right? So look at line three. I'm saying, make sure that there's only one book visible. That's the functional assertion. And once that's the case, I can then say, now, do the visual test. So that makes it really, really powerful.

7. Dynamic Content and Cross-Platform Testing

Short description:

You can verify dynamic content with visual testing using Apple tools. Different match levels provide flexibility and allow the detection of layout patterns on the page. Running tests across multiple platforms is made possible with Applitudes' ultra-fast grid, which blasts the application state across all devices. This eliminates the need for physical devices and allows simultaneous testing on various browsers and devices. The grid can also be used to verify components, ensuring visual perfection for date pickers, fields, buttons, and more.

All right? I can also do dynamic content here. So you wouldn't think you could do dynamic, verify dynamic content with visual testing, but you can with Apple tools, right? So this is a match level. There's several different match levels. So this makes it very verbose and flexible. But this layout is saying, I want you to use machine learning in a different way to just detect the layout of my page.

So for example, this first screenshot, this is the books in one order. The second screenshot is the books in another order. Because I said to verify this using the layout algorithm, it's just going to detect the patterns on the page. Very popular for like news sites. When I used to work at Twitter, we used that match level for the tweets. We don't know what they're gonna say, but we wanna make sure that structurally, the page is okay, things aren't overlapping or bleeding off the edge, right? So even though those books are in different order, this still passes.

And then the final thing I wanna show you is the ability to run this across multiple platforms. So Applitudes has this ultra fast grid that works beautifully with eyes. And what this does is allow you to run your test across any browser or device that you want to simultaneously. And the way that it does this is it grabs the state of your application and then blasts it across all of those devices. So you don't need the device phone, you don't have to deal with that at all. You don't even have to change your test. You simply add line, starting on line four, we have an array that basically says, here's what I want. I want Firefox common IE in these dimensions. I want iPhone 10 in landscape and portrait mode. I want Galaxy. And this all runs lightning fast simultaneously. Pretty neat. And so that's what that would look like on the grid. Notice here, bunch of tests run here, all the different devices. Very, very, very cool. Also works for components. So if you have Storybook for Angular, Vue, React, this can verify your components and make sure your date pickers, your field, your buttons, everything remains visually perfect as the rest of your teams are gonna use these things. All right? So there you have it. That is visual testing in a nutshell.

QnA

Visual Testing Q&A

Short description:

Yuval asked if visual regression should be applied to all pages or just critical ones. It is recommended to use visual testing on stable pages. However, for pages undergoing frequent changes, you can use the API creatively to ignore certain sections or treat them as dynamic content. To minimize false positives, you can run tests locally and flag intentional changes with a thumbs up or thumbs down. Manuel asked about visually testing pages with dynamic content. Applitools' dashboard allows you to annotate images and mix different match levels, including a dynamic match level.

What questions can I answer for you?

Hello, Alex. How are you?

I'm good. I'm trying to survive. I'm trying to survive. How are you?

Yeah, I'm good. I'm good. Hanging in there.

Cool. We've got quite a few questions. I'm gonna probably butcher the names of the people who asked the question. So Yuval wants to know, should I have visual regression on all my pages or just some critical ones? If so, which ones are critical? That's a good question. So you should have visual testing on your stable-type pages, right? So if you have an application that is undergoing a lot of maintenance, for example, this specific page, you're constantly adding new components to it and it's not quite stable just yet, that might not be a good candidate for visual testing. Now, you could use the API in some creative ways and say, kind of, ignore this active section of the page or verify it as like dynamic content that's going to change or whatever. So you could do it, but my recommendation is to use it on your pages that are most stable.

Okay, that's really good, that's really good. They have a follow up question that says, how do I minimize false positive? Is there a way to flag changes as intentional beforehand? Oh, that's good. So I would say it's not a beforehand kind of thing, but it could be before it's like, actually part of your CI or something like that, right? So I could run something locally. Let's say I develop, I change a feature, right? That changes the website. And so I know that the baseline is going to change. So I can run that locally. And then when it fails, I can just do a thumbs up. That's all it takes, a thumbs up or thumbs down to say intentional or not. And I can say, this is an intentional change, check that in. And then when it runs as part of the bigger suite, it'll be okay.

Okay, that makes a lot of sense. That makes a lot of sense. Manuel wants to know, how do you visually test the page that loads dynamic content, like a changing list of users, for example? Right, so you can do this on the dashboard. So Abilitudes has a dashboard that allow you to annotate the image. So you can mix and match the different match levels, right? So I talked about the dynamic match level. I talked about the default one, which was just kind of strict mode.

Dynamic Content and Annotating Elements

Short description:

You can mix and match dynamic and regular verification. Annotate dynamic elements programmatically using CSS selectors or on the dashboard. For example, the status bar in mobile testing can be ignored programmatically. This makes writing new mobile tests easier.

You can mix and match. You can say, for example, like that list, this part is dynamic, everything else, I want you to verify regularly. Some people use this a lot for like dates, timestamps, those sorts of things that are constantly changing on the site so you can annotate it. And you can do that on the dashboard or you can do it programmatically by giving it like the CSS selector of that element so I can say list of users, whatever that idea or whatever it is, I can specify that programmatically so I don't have to go back and annotate it on the dashboard.

A great way that I use that is with mobile testing. So on the top of your mobile is the status bar, right? With your notifications and your time, everything up there is dynamic. So programmatically I say always ignore that specific status bar element and so when I write new mobile test, it already ignores that part. Oh, that's actually quite interesting. That's quite interesting. Thank you.

Appy Tools: Flaky Test Results and Accessibility

Short description:

Appy tools ensures test results are not flaky by using machine learning, which improves accuracy over time. Snapshot testing, although still viable, is more fragile and requires additional maintenance. Appy tools also offers accessibility features, including a contrast tester. An on-premise option is available for customers with more advanced needs.

How does Appy tools ensure that the test results are not flaky? Yeah, so it uses machine learning which is different than the pixel-to-pixel. So we have like a 99.5% accuracy and because it's using machine learning, it gets better and better, you know, over time as it learns more about the things that are kind of considered flaky or not.

Okay, okay. So we've got a lot of questions around Appy tools. I am gonna continue with that because it's something new for me at least. I haven't used it before and I wanna know more. Considering the capabilities and benefits of visual testing, is there still a benefit to snapshot testing or would incorporating Appy tools renders those obsolete?

Right, so if you're thinking of snapshot testing, like I know a couple of ways to do that. One is like capturing the DOM or whatever, the entire state of the application and then comparing that. So you have these textual type of changes, right? That in my opinion is a little bit more fragile than the picture one, right? Because I might change the underlying code without changing the visual aspects of the page, right? So it just leads to more maintenance to do this snapshot testing but that's still a viable option.

Okay, okay. That makes sense, that makes sense. I've got a question that I think is kind of related so I'm gonna jump to that. How about accessibility tests? Is there any possibility to automate them?

Yeah, so Apple IIs also has accessibility features and this is great. Like it's a great way to use visual testing outside of the box. So it has this contrast tester. So it's looking to make sure that all of your elements are adhering to like the AA or you know, AA guidelines or whatever, as far as the accessibility. So that's something you can turn on or turn off.

Okay, okay, that makes a lot of sense. That makes a lot of sense. One more question, it's a quick one and then we can, we can let you go to your speaker's room. So do you have an on-premise option for Applitools server?

Yes, so the on-premise option, of course that was more costly. There's a free plan for everyone. So everyone can use this for free. If you need like more features, more, you know, capabilities you can upgrade, but then the on-premise option is more, but yes, it is available in our customers that are like banks, healthcare. They use the on-premise option.

Okay, okay. Thank you. You've had so many questions, Angie. It's really, really hard for me to ask you all of those. So for the people which I've not asked the questions, I'm really sorry, but Angie has a speaker's room on spatial.chat. The link to join is on your timeline and you can join Angie in the room and ask her all the questions that I haven't asked. So thank you, Angie, for joining us today. It was really, really good to see your talk and get the chance to talk to you a little bit. Thank you so much. Sure thing. Bye, Alec.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Network Requests with Cypress
TestJS Summit 2021TestJS Summit 2021
33 min
Network Requests with Cypress
Top Content
Cecilia Martinez, a technical account manager at Cypress, discusses network requests in Cypress and demonstrates commands like cydot request and SCI.INTERCEPT. She also explains dynamic matching and aliasing, network stubbing, and the pros and cons of using real server responses versus stubbing. The talk covers logging request responses, testing front-end and backend API, handling list length and DOM traversal, lazy loading, and provides resources for beginners to learn Cypress.
Testing Pyramid Makes Little Sense, What We Can Use Instead
TestJS Summit 2021TestJS Summit 2021
38 min
Testing Pyramid Makes Little Sense, What We Can Use Instead
Top Content
Featured Video
Gleb Bahmutov
Roman Sandler
2 authors
The testing pyramid - the canonical shape of tests that defined what types of tests we need to write to make sure the app works - is ... obsolete. In this presentation, Roman Sandler and Gleb Bahmutov argue what the testing shape works better for today's web applications.
Full-Circle Testing With Cypress
TestJS Summit 2022TestJS Summit 2022
27 min
Full-Circle Testing With Cypress
Top Content
Cypress is a powerful tool for end-to-end testing and API testing. It provides instant feedback on test errors and allows tests to be run inside the browser. Cypress enables testing at both the application and network layers, making it easier to reach different edge cases. With features like AppActions and component testing, Cypress allows for comprehensive testing of individual components and the entire application. Join the workshops to learn more about full circle testing with Cypress.
Test Effective Development
TestJS Summit 2021TestJS Summit 2021
31 min
Test Effective Development
Top Content
This Talk introduces Test Effective Development, a new approach to testing that aims to make companies more cost-effective. The speaker shares their personal journey of improving code quality and reducing bugs through smarter testing strategies. They discuss the importance of finding a balance between testing confidence and efficiency and introduce the concepts of isolated and integrated testing. The speaker also suggests different testing strategies based on the size of the application and emphasizes the need to choose cost-effective testing approaches based on the specific project requirements.
Playwright Test Runner
TestJS Summit 2021TestJS Summit 2021
25 min
Playwright Test Runner
Top Content
The Playwright Test Runner is a cross-browser web testing framework that allows you to write tests using just a few lines of code. It supports features like parallel test execution, device emulation, and different reporters for customized output. Code-Gen is a new feature that generates code to interact with web pages. Playwright Tracing provides a powerful tool for debugging and analyzing test actions, with the ability to explore trace files using TraceViewer. Overall, Playwright Test offers installation, test authoring, debugging, and post-mortem debugging capabilities.
Everyone Can Easily Write Tests
TestJS Summit 2023TestJS Summit 2023
21 min
Everyone Can Easily Write Tests
Playwright is a reliable end-to-end testing tool for modern web apps that provides one API, full isolation, fast execution, and supports multiple languages. It offers features like auto-weighting, retrying assertions, seamless testing of iframes and shadow DOM, test isolation, parallelism, and scalability. Playwright provides tools like VS Code extension, UiMode, and Trace Viewer for writing, debugging, and running tests. Effective tests prioritize user-facing attributes, use playwright locators and assertions, and avoid testing third-party dependencies. Playwright simplifies testing by generating tests, providing code generation and UI mode, and allows for easy running and debugging of tests. It helps in fixing failed tests and analyzing DOM changes, fixing locator mismatches, and scaling tests. Playwright is open source, free, and continuously growing.

Workshops on related topic

Designing Effective Tests With React Testing Library
React Summit 2023React Summit 2023
151 min
Designing Effective Tests With React Testing Library
Top Content
Featured Workshop
Josh Justice
Josh Justice
React Testing Library is a great framework for React component tests because there are a lot of questions it answers for you, so you don’t need to worry about those questions. But that doesn’t mean testing is easy. There are still a lot of questions you have to figure out for yourself: How many component tests should you write vs end-to-end tests or lower-level unit tests? How can you test a certain line of code that is tricky to test? And what in the world are you supposed to do about that persistent act() warning?
In this three-hour workshop we’ll introduce React Testing Library along with a mental model for how to think about designing your component tests. This mental model will help you see how to test each bit of logic, whether or not to mock dependencies, and will help improve the design of your components. You’ll walk away with the tools, techniques, and principles you need to implement low-cost, high-value component tests.
Table of contents- The different kinds of React application tests, and where component tests fit in- A mental model for thinking about the inputs and outputs of the components you test- Options for selecting DOM elements to verify and interact with them- The value of mocks and why they shouldn’t be avoided- The challenges with asynchrony in RTL tests and how to handle them
Prerequisites- Familiarity with building applications with React- Basic experience writing automated tests with Jest or another unit testing framework- You do not need any experience with React Testing Library- Machine setup: Node LTS, Yarn
Detox 101: How to write stable end-to-end tests for your React Native application
React Summit 2022React Summit 2022
117 min
Detox 101: How to write stable end-to-end tests for your React Native application
Top Content
Workshop
Yevheniia Hlovatska
Yevheniia Hlovatska
Compared to unit testing, end-to-end testing aims to interact with your application just like a real user. And as we all know it can be pretty challenging. Especially when we talk about Mobile applications.
Tests rely on many conditions and are considered to be slow and flaky. On the other hand - end-to-end tests can give the greatest confidence that your app is working. And if done right - can become an amazing tool for boosting developer velocity.
Detox is a gray-box end-to-end testing framework for mobile apps. Developed by Wix to solve the problem of slowness and flakiness and used by React Native itself as its E2E testing tool.
Join me on this workshop to learn how to make your mobile end-to-end tests with Detox rock.
Prerequisites- iOS/Android: MacOS Catalina or newer- Android only: Linux- Install before the workshop
API Testing with Postman Workshop
TestJS Summit 2023TestJS Summit 2023
48 min
API Testing with Postman Workshop
Top Content
WorkshopFree
Pooja Mistry
Pooja Mistry
In the ever-evolving landscape of software development, ensuring the reliability and functionality of APIs has become paramount. "API Testing with Postman" is a comprehensive workshop designed to equip participants with the knowledge and skills needed to excel in API testing using Postman, a powerful tool widely adopted by professionals in the field. This workshop delves into the fundamentals of API testing, progresses to advanced testing techniques, and explores automation, performance testing, and multi-protocol support, providing attendees with a holistic understanding of API testing with Postman.
1. Welcome to Postman- Explaining the Postman User Interface (UI)2. Workspace and Collections Collaboration- Understanding Workspaces and their role in collaboration- Exploring the concept of Collections for organizing and executing API requests3. Introduction to API Testing- Covering the basics of API testing and its significance4. Variable Management- Managing environment, global, and collection variables- Utilizing scripting snippets for dynamic data5. Building Testing Workflows- Creating effective testing workflows for comprehensive testing- Utilizing the Collection Runner for test execution- Introduction to Postbot for automated testing6. Advanced Testing- Contract Testing for ensuring API contracts- Using Mock Servers for effective testing- Maximizing productivity with Collection/Workspace templates- Integration Testing and Regression Testing strategies7. Automation with Postman- Leveraging the Postman CLI for automation- Scheduled Runs for regular testing- Integrating Postman into CI/CD pipelines8. Performance Testing- Demonstrating performance testing capabilities (showing the desktop client)- Synchronizing tests with VS Code for streamlined development9. Exploring Advanced Features - Working with Multiple Protocols: GraphQL, gRPC, and more
Join us for this workshop to unlock the full potential of Postman for API testing, streamline your testing processes, and enhance the quality and reliability of your software. Whether you're a beginner or an experienced tester, this workshop will equip you with the skills needed to excel in API testing with Postman.
Monitoring 101 for React Developers
React Summit US 2023React Summit US 2023
107 min
Monitoring 101 for React Developers
Top Content
WorkshopFree
Lazar Nikolov
Sarah Guthals
2 authors
If finding errors in your frontend project is like searching for a needle in a code haystack, then Sentry error monitoring can be your metal detector. Learn the basics of error monitoring with Sentry. Whether you are running a React, Angular, Vue, or just “vanilla” JavaScript, see how Sentry can help you find the who, what, when and where behind errors in your frontend project. 
Workshop level: Intermediate
Testing Web Applications Using Cypress
TestJS Summit - January, 2021TestJS Summit - January, 2021
173 min
Testing Web Applications Using Cypress
Top Content
Workshop
Gleb Bahmutov
Gleb Bahmutov
This workshop will teach you the basics of writing useful end-to-end tests using Cypress Test Runner.
We will cover writing tests, covering every application feature, structuring tests, intercepting network requests, and setting up the backend data.
Anyone who knows JavaScript programming language and has NPM installed would be able to follow along.
Best Practices for Writing and Debugging Cypress Tests
TestJS Summit 2023TestJS Summit 2023
148 min
Best Practices for Writing and Debugging Cypress Tests
Top Content
Workshop
Filip Hric
Filip Hric
You probably know the story. You’ve created a couple of tests, and since you are using Cypress, you’ve done this pretty quickly. Seems like nothing is stopping you, but then – failed test. It wasn’t the app, wasn’t an error, the test was… flaky? Well yes. Test design is important no matter what tool you will use, Cypress included. The good news is that Cypress has a couple of tools behind its belt that can help you out. Join me on my workshop, where I’ll guide you away from the valley of anti-patterns into the fields of evergreen, stable tests. We’ll talk about common mistakes when writing your test as well as debug and unveil underlying problems. All with the goal of avoiding flakiness, and designing stable test.