AI-Powered E2E UI Testing: Faster Creation, Easier Maintenance

Rate this content
Bookmark
GithubProject website

This session will guide you through the advanced field of AI- E2E UI Testing, focusing on how it speeds up test creation and simplifies test maintenance. We'll begin by examining the challenges in generating E2E tests with AI, noting how this differs from the simpler task of creating unit tests.
The discussion will then shift to a solution that addresses these challenges effectively. Prepare to see a live demonstration where we'll craft an E2E test using popular tools like Cypress or Playwright. To wrap up, we'll look into how AI comes close to fully automating the maintenance of our test suites, ensuring they stay up-to-date with minimal manual intervention.

This talk has been presented at JSNation US 2024, check out the latest edition of this JavaScript Conference.

FAQ

The two best practices are using meaningful data test IDs and relying on the page objects model pattern to make tests cleaner and more modular.

You can follow Daniel Ostrovsky on Twitter, Medium, and YouTube by scanning the provided barcode. His username is nanduh81.

The main topic of the presentation is AI-powered end-to-end testing for user interfaces.

The continue extension is used for managing code directly in the IDE, offering features like auto-completion and helping in generating end-to-end tests.

AI-powered UI testing faces challenges due to the abstraction layers between the source code and the rendered UI, unlike unit tests where the source code is directly available.

AI can effectively generate tests for common applications like e-commerce websites due to existing knowledge. However, for niche applications, it struggles without specific guidance as it lacks access to proprietary systems.

The application used for the demo is a Pokemon application, which includes features like filtering by name or type, and navigation to Pokemon details pages.

Daniel recommends using an in-browser extension called end-to-end test helper and an IDE extension called continue, which offers features like powerful auto-completion.

Daniel Ostrovsky is a full-cycle developer expert with over 25 years of experience. He is a public speaker, writer, open-source contributor, and technical mentor.

Meaningful data test IDs help AI understand the purpose of each actionable item, making it easier to generate and execute relevant tests for UI elements.

Daniel Ostrovsky
Daniel Ostrovsky
10 min
21 Nov, 2024

Comments

Sign in or register to post your comment.
Video Summary and Transcription
Hello, everyone. Today we are going to explore AI-powered end-to-end testing. Unlike unit tests, UI testing has a huge layer of obstructions between the source code and the rendered UI. The source code includes HTML, CSS, and TypeScript, which are transpiled into JavaScript and bundled with tools like Webpack. AI can generate tests effectively for standard websites or blogs, but it may struggle with niche applications behind strict authorization or on-premise tools. AI-powered end-to-end testing for complex scenarios requires our guidance. We use meaningful data test IDs and follow the page objects model pattern. Additionally, we rely on useful tools like the end-to-end test helper in-browser extension and the continue IDE extension. Now, let's proceed to the demo, where we will create tests for the Pokemon application, including the ability to filter by name or type. We will navigate to the Pokemon details page and use our extension to manage settings and prompts. Additionally, we will create the details page object together and generate the test file. The Pokemon details page has 105 elements. We can view the elements for debugging purposes, including page object name, Pokemon details page, and system message. We will copy the page object and save it to a file. We need to make it exportable. Then, we will use the extension to create an end-to-end test and pass the context. I will use all the open files, including the page objects and the test case itself. I will send them to EI along with the predefined prompt. There is a system message and additional information we need to be aware of. The test runs successfully, and that concludes this part.

1. Introduction to AI-powered end-to-end testing

Short description:

Hello, everyone. Today we are going to explore AI-powered end-to-end testing. Unlike unit tests, UI testing has a huge layer of obstructions between the source code and the rendered UI. The source code includes HTML, CSS, and TypeScript, which are transpiled into JavaScript and bundled with tools like Webpack. The rendered UI is obtained by rendering engines like React and Angular. AI can generate tests effectively for standard websites or blogs, but it may struggle with niche applications behind strict authorization or on-premise tools.

Hello, everyone. My name is Daniel Ostrovsky, and today we are going to explore AI-powered end-to-end testing. But before we dive in, go ahead and scan this barcode to follow me on Twitter, Medium, and YouTube. You can find me as nanduh81. Thank you.

And let's continue. So first of all, a little bit about myself. I'm a full-cycle developer expert. I have more than 25 years of experience. I'm a public speaker, public writer on multiple languages. I'm an open-source contributor, technical mentor, and again, scan the barcode and follow me.

So let's jump right into the challenges that we face in when we're trying to test UI with AI. Okay, now, unlike unit tests, where we can send the source code directly into a LLM, which makes it easy for the model to predict how to test should look like, UI testing is different. So for unit tests, tools like, you know, Copilot and others can generate a test in seconds because they have the full context, source code, and logic that needs to be tested. And in most cases, it works perfectly because the code and context are clear. But when it comes to UI testing, there is a huge layer of obstructions between the source code and what we're actually testing, the rendered UI.

So here's the path. We start our code with actually includes HTML, CSS, and TypeScript, right, that get transpiled into the JavaScript. Then it bundles with tools like Webpack, Write and others. It just make all the transpolation, you know, the bundle and other things. Okay. Unifications and everything. And then it goes to renders engines like React and Angular that takes over the browser. OK, then finally, everything will be rendered by browser. Now only at that point, we do get the actual visual elements we need to test. OK. So where's the source code and what actually we need to test? It's a huge difference. So if we're talking about the standard e-commerce website or blog or something similar, AI has a lot of existing knowledge to get from because it's trained on similar publicly available data. So when we send data to an LLM, 80% of the time, it can generate tests effectively because, you know, AI is already familiar with the existing patterns. OK, but what if you are working on something niche, like an internal application behind a strict authorization or a tool that only works on premise? The LLM doesn't have access to these systems, and therefore, it doesn't understand the specific flows, user interactions, and functionality.

2. Best Practices and Tooling for AI-powered Testing

Short description:

AI-powered end-to-end testing for complex scenarios requires our guidance. We use meaningful data test IDs and follow the page objects model pattern. Additionally, we rely on useful tools like the end-to-end test helper in-browser extension and the continue IDE extension. Now, let's proceed to the demo, where we will create tests for the Pokemon application, including the ability to filter by name or type.

It's not like e-commerce where adding an item to the cart and checking out is predictable. What if you're testing some kind of configuration tool for managing hundreds of gas stations in the middle of the ocean? So AI won't know how to handle it without our guidance. So that's where we need to help AI to help us.

And for that, we follow two key best practices. First, use meaningful data test IDs. In our case, we use data CY since we're working with Cypress. But the principle remains the same. Elements like buttons and input fields should have clear descriptive names, such as next button, username input, and so on. This helps the AI understand the purpose of each actionable item. Second, we rely on the page objects model pattern. While the AI will generate this page for us, so we don't need to create it and it'll show how we're doing it, it's crucial to maintain and manage them. This approach makes our tests cleaner and more modular.

Lastly, we will need tools. One of them is an in-browser extension called end-to-end test helper. You can scan this barcode again, barcode always and check it out. Contribute, please, or just explore the documentation. And please take a look and consider a contribution. I really would love to have more hands in the room. For the third tool, we use an IDE extension called continue. It's a fantastic extension, by the way, similar to tools like Copilot and others that help you manage and work with your code directly in UID. It offers powerful auto-completion and many other cool features that, by the way, Copilot doesn't have. It's open source, so definitely check it out. It's a great tool to have in your toolbox. Great, all right.

So let's move to the demo, to the real demo. Give me a second. Here we go. So, this is the application that we will create the tests for. This is the Pokemon application. We have the Pokemon list ability to filter by name or filter by type.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Building Better Websites with Remix
React Summit Remote Edition 2021React Summit Remote Edition 2021
33 min
Building Better Websites with Remix
Top Content
Remix is a web framework built on React Router that focuses on web fundamentals, accessibility, performance, and flexibility. It delivers real HTML and SEO benefits, and allows for automatic updating of meta tags and styles. It provides features like login functionality, session management, and error handling. Remix is a server-rendered framework that can enhance sites with JavaScript but doesn't require it for basic functionality. It aims to create quality HTML-driven documents and is flexible for use with different web technologies and stacks.
Speeding Up Your React App With Less JavaScript
React Summit 2023React Summit 2023
32 min
Speeding Up Your React App With Less JavaScript
Top Content
Watch video: Speeding Up Your React App With Less JavaScript
Mishko, the creator of Angular and AngularJS, discusses the challenges of website performance and JavaScript hydration. He explains the differences between client-side and server-side rendering and introduces Quik as a solution for efficient component hydration. Mishko demonstrates examples of state management and intercommunication using Quik. He highlights the performance benefits of using Quik with React and emphasizes the importance of reducing JavaScript size for better performance. Finally, he mentions the use of QUIC in both MPA and SPA applications for improved startup performance.
Full Stack Documentation
JSNation 2022JSNation 2022
28 min
Full Stack Documentation
Top Content
The Talk discusses the shift to full-stack frameworks and the challenges of full-stack documentation. It highlights the power of interactive tutorials and the importance of user testing in software development. The Talk also introduces learn.svelte.dev, a platform for learning full-stack tools, and discusses the roadmap for SvelteKit and its documentation.
Network Requests with Cypress
TestJS Summit 2021TestJS Summit 2021
33 min
Network Requests with Cypress
Top Content
Cecilia Martinez, a technical account manager at Cypress, discusses network requests in Cypress and demonstrates commands like cydot request and SCI.INTERCEPT. She also explains dynamic matching and aliasing, network stubbing, and the pros and cons of using real server responses versus stubbing. The talk covers logging request responses, testing front-end and backend API, handling list length and DOM traversal, lazy loading, and provides resources for beginners to learn Cypress.
Testing Pyramid Makes Little Sense, What We Can Use Instead
TestJS Summit 2021TestJS Summit 2021
38 min
Testing Pyramid Makes Little Sense, What We Can Use Instead
Top Content
Featured Video
Gleb Bahmutov
Roman Sandler
2 authors
The testing pyramid - the canonical shape of tests that defined what types of tests we need to write to make sure the app works - is ... obsolete. In this presentation, Roman Sandler and Gleb Bahmutov argue what the testing shape works better for today's web applications.
SolidJS: Why All the Suspense?
JSNation 2023JSNation 2023
28 min
SolidJS: Why All the Suspense?
Top Content
Suspense is a mechanism for orchestrating asynchronous state changes in JavaScript frameworks. It ensures async consistency in UIs and helps avoid trust erosion and inconsistencies. Suspense boundaries are used to hoist data fetching and create consistency zones based on the user interface. They can handle loading states of multiple resources and control state loading in applications. Suspense can be used for transitions, providing a smoother user experience and allowing prioritization of important content.

Workshops on related topic

Designing Effective Tests With React Testing Library
React Summit 2023React Summit 2023
151 min
Designing Effective Tests With React Testing Library
Top Content
Featured Workshop
Josh Justice
Josh Justice
React Testing Library is a great framework for React component tests because there are a lot of questions it answers for you, so you don’t need to worry about those questions. But that doesn’t mean testing is easy. There are still a lot of questions you have to figure out for yourself: How many component tests should you write vs end-to-end tests or lower-level unit tests? How can you test a certain line of code that is tricky to test? And what in the world are you supposed to do about that persistent act() warning?
In this three-hour workshop we’ll introduce React Testing Library along with a mental model for how to think about designing your component tests. This mental model will help you see how to test each bit of logic, whether or not to mock dependencies, and will help improve the design of your components. You’ll walk away with the tools, techniques, and principles you need to implement low-cost, high-value component tests.
Table of contents- The different kinds of React application tests, and where component tests fit in- A mental model for thinking about the inputs and outputs of the components you test- Options for selecting DOM elements to verify and interact with them- The value of mocks and why they shouldn’t be avoided- The challenges with asynchrony in RTL tests and how to handle them
Prerequisites- Familiarity with building applications with React- Basic experience writing automated tests with Jest or another unit testing framework- You do not need any experience with React Testing Library- Machine setup: Node LTS, Yarn
Detox 101: How to write stable end-to-end tests for your React Native application
React Summit 2022React Summit 2022
117 min
Detox 101: How to write stable end-to-end tests for your React Native application
Top Content
WorkshopFree
Yevheniia Hlovatska
Yevheniia Hlovatska
Compared to unit testing, end-to-end testing aims to interact with your application just like a real user. And as we all know it can be pretty challenging. Especially when we talk about Mobile applications.
Tests rely on many conditions and are considered to be slow and flaky. On the other hand - end-to-end tests can give the greatest confidence that your app is working. And if done right - can become an amazing tool for boosting developer velocity.
Detox is a gray-box end-to-end testing framework for mobile apps. Developed by Wix to solve the problem of slowness and flakiness and used by React Native itself as its E2E testing tool.
Join me on this workshop to learn how to make your mobile end-to-end tests with Detox rock.
Prerequisites- iOS/Android: MacOS Catalina or newer- Android only: Linux- Install before the workshop
API Testing with Postman Workshop
TestJS Summit 2023TestJS Summit 2023
48 min
API Testing with Postman Workshop
Top Content
WorkshopFree
Pooja Mistry
Pooja Mistry
In the ever-evolving landscape of software development, ensuring the reliability and functionality of APIs has become paramount. "API Testing with Postman" is a comprehensive workshop designed to equip participants with the knowledge and skills needed to excel in API testing using Postman, a powerful tool widely adopted by professionals in the field. This workshop delves into the fundamentals of API testing, progresses to advanced testing techniques, and explores automation, performance testing, and multi-protocol support, providing attendees with a holistic understanding of API testing with Postman.
1. Welcome to Postman- Explaining the Postman User Interface (UI)2. Workspace and Collections Collaboration- Understanding Workspaces and their role in collaboration- Exploring the concept of Collections for organizing and executing API requests3. Introduction to API Testing- Covering the basics of API testing and its significance4. Variable Management- Managing environment, global, and collection variables- Utilizing scripting snippets for dynamic data5. Building Testing Workflows- Creating effective testing workflows for comprehensive testing- Utilizing the Collection Runner for test execution- Introduction to Postbot for automated testing6. Advanced Testing- Contract Testing for ensuring API contracts- Using Mock Servers for effective testing- Maximizing productivity with Collection/Workspace templates- Integration Testing and Regression Testing strategies7. Automation with Postman- Leveraging the Postman CLI for automation- Scheduled Runs for regular testing- Integrating Postman into CI/CD pipelines8. Performance Testing- Demonstrating performance testing capabilities (showing the desktop client)- Synchronizing tests with VS Code for streamlined development9. Exploring Advanced Features - Working with Multiple Protocols: GraphQL, gRPC, and more
Join us for this workshop to unlock the full potential of Postman for API testing, streamline your testing processes, and enhance the quality and reliability of your software. Whether you're a beginner or an experienced tester, this workshop will equip you with the skills needed to excel in API testing with Postman.
Monitoring 101 for React Developers
React Summit US 2023React Summit US 2023
107 min
Monitoring 101 for React Developers
Top Content
WorkshopFree
Lazar Nikolov
Sarah Guthals
2 authors
If finding errors in your frontend project is like searching for a needle in a code haystack, then Sentry error monitoring can be your metal detector. Learn the basics of error monitoring with Sentry. Whether you are running a React, Angular, Vue, or just “vanilla” JavaScript, see how Sentry can help you find the who, what, when and where behind errors in your frontend project. 
Workshop level: Intermediate
Building a Shopify App with React & Node
React Summit Remote Edition 2021React Summit Remote Edition 2021
87 min
Building a Shopify App with React & Node
Top Content
WorkshopFree
Jennifer Gray
Hanna Chen
2 authors
Shopify merchants have a diverse set of needs, and developers have a unique opportunity to meet those needs building apps. Building an app can be tough work but Shopify has created a set of tools and resources to help you build out a seamless app experience as quickly as possible. Get hands on experience building an embedded Shopify app using the Shopify App CLI, Polaris and Shopify App Bridge.We’ll show you how to create an app that accesses information from a development store and can run in your local environment.
Testing Web Applications Using Cypress
TestJS Summit - January, 2021TestJS Summit - January, 2021
173 min
Testing Web Applications Using Cypress
Top Content
WorkshopFree
Gleb Bahmutov
Gleb Bahmutov
This workshop will teach you the basics of writing useful end-to-end tests using Cypress Test Runner.
We will cover writing tests, covering every application feature, structuring tests, intercepting network requests, and setting up the backend data.
Anyone who knows JavaScript programming language and has NPM installed would be able to follow along.