Predictive Testing in JavaScript with Machine Learning

Rate this content
Bookmark

This talk will cover how we can apply machine learning to software testing including in Javascript, to help reduce the number of tests that need to be run.

We can use predictive machine learning model to make an informed decision to rule out tests that are extremely unlikely to uncover an issue. This opens up a new, more efficient way of selecting tests.

This talk has been presented at TestJS Summit 2021, check out the latest edition of this JavaScript Conference.

FAQ

Machine learning can improve software testing by automating test case generation, prioritizing the most relevant tests to run, and enhancing code coverage analysis. It can analyze historical testing data to predict and suggest the most effective tests for new code changes, thereby reducing the time and effort required in manual testing processes.

While AI can significantly enhance testing efficiency by automating many aspects, it cannot fully replace manual testing. Human oversight is crucial for handling complex test scenarios, understanding user experiences, and making strategic decisions based on the context that AI might not fully grasp.

AI contributes to regression testing by analyzing past test cases and code changes to predict the likelihood of new changes causing failures. This allows testers to focus on high-risk areas more effectively, improving the quality of the software and efficiency of the testing process.

Integrating AI with JavaScript testing frameworks like Jest or Jasmine can help automate the generation of test cases, suggest relevant tests based on code changes, and improve regression testing. This integration enhances test efficiency, coverage, and accuracy, leading to higher quality applications.

TensorFlow.JS plays a critical role in AI-based testing by allowing the integration of machine learning models directly into JavaScript applications. It enables the execution of models in the browser or on Node.js, facilitating on-the-fly test case generation and other AI-driven testing processes directly in JavaScript environments.

Predictive testing in JavaScript with machine learning involves using AI to generate test cases automatically, select the most relevant tests for specific code changes, and achieve higher code coverage. This approach leverages historical data and machine learning algorithms to optimize the testing process, making it more efficient and effective.

Real-world applications of AI in software testing include automated test case generation, integration with testing frameworks like Selenium for improved automation, and the use of AI tools like GitHub Copilot for code suggestions. AI helps in identifying critical tests, reducing redundant tests, and ensuring comprehensive code coverage.

Shivay Lamba
Shivay Lamba
18 min
19 Nov, 2021

Comments

Sign in or register to post your comment.
Video Summary and Transcription
This Talk explores the benefits of introducing machine learning to software testing, including automating test case generation and achieving close to 100% code coverage. AI is being used to automate test generation, improve regression testing, and make predictions in automation testing. Machine learning enables predictive testing by selecting tests that are more likely to uncover issues in code changes. AI-based tools are being used to generate automated tests, improve code coverage, and intelligently select tests. Companies are relying on dedicated testers and using historical code changes and test cases to generate specific test cases for relevant code changes.

1. Introduction to Predictive Testing in JavaScript

Short description:

Hello, everyone! Welcome to TestJS Summit 2021. I'm Shivaay, and I'll be presenting on predictive testing in JavaScript with machine learning. Let's get started!

Hello, everyone, welcome to TestJS Summit 2021, and I'm Shivaay, who is going to be presenting a topic on predictive testing in JavaScript with machine learning. So a very quick introduction about myself, I'm Shivaay and I'm currently a TensorFlow.JS Working Group member. I'm a Google Server Code mentor at TensorFlow and this is my third GitNation conference talk this year. Previously, I've also given a talk at Node.js conference and also React Advanced this year. So really excited to be presenting yet another talk at GitNation, specifically at TestJS Summit. So without wasting any further time, let's get started.

2. Introducing Machine Learning to Software Testing

Short description:

Machine learning is being used in various industries, including software development. However, software testing has not fully utilized AI and machine learning. This talk discusses the benefits of introducing machine learning to software testing. AI can automate test case generation, determine which tests are most important, and achieve close to 100% code coverage. AI can also improve automation testing frameworks like Selenium by identifying and resolving issues.

Now, first one, machine learning is really everywhere today. There is no doubt in that particular fact. You can see machine learning being used in healthcare, education, but also when it comes to things within software development itself. Machine learning today is being used to do a lot of different things. For example, we have seen how GitHub Copilot is being used today to auto-generate or auto-suggest new code. It's also being used in MLOps, in different forms of operations in DevOps to improve the DevOps cycles. And that sort of makes it really important to also use it in other areas where we traditionally might not think that it might be able to be used. So that's what makes machine learning really powerful today. And that's why, why not just think about introducing machine learning to software testing?

Software testing has traditionally been all about being able to write test cases, being able to sort of understand fundamentally within the entire software engineering lifecycle that how can we actually make our softwares more and more productive and make them error free. So what that means is that ideally what happens if you have a software tester or a QA analyst, they'll be going through the entire code and writing unit test cases. And of course, we have the entire pipeline of how the testing actually starts off, where first we write some code, then we prepare the unit tests, we have integration tests. And based on this, once the test cases are passed and our code passes through all these different test cases from both the unit and the integration test, then we finally put our code into production. We also use things such as regression testing to be able to see how is new code actually impacting the code that has been written prior to or let's say, some of the older code bases and how this code is actually getting affected. So all of these different processes involve basically either you can use manual testing or you also have a lot of different automation testing tools as well for example like Selenium. And these traditionally haven't really used machine learning. It has more to do with actually writing these test cases if you're doing manual testing or even if you're using automation testing tools such as Selenium to be able to configure them so that they are able to go through your application. And generally, the software testing has not really seen a lot of use of AI but this talk specifically will talk about how we can introduce and what are the benefits of actually adding machine learning or AI to software testing.

So, write about some of the great scenarios where AI could be actually used in testing. So the first one is the automated test case generation. So a lot of time what happens is that we spend a lot of time in actually creating test cases. Now that could be behavior driven programming or behavior driven development where we're creating the test cases before writing the code itself or let's say we have written a particular function and we write a unit test case for it. How can we actually use AI to generate test cases on fly by just evaluating the code by going through the code? Where an AI model just understands whatever code has been written, whatever code changes have been made to the code and then automatically generate test cases without any manual intervention? Then, essentially, being able to find out which particular tests to actually run that are more important that will essentially save time and sort of the entire process of the software testing time where we are only running the most important test cases. I'll also be coming into more deeper discussions specifically on the second point and then how we can actually use machine learning to not only test frontend based UI testing specifically on, say, JavaScript, but also how we can use machine learning to actually test out backend APIs. That's also one of the fields where today a lot of AI-based software testing tools are being worked on. And then how can we achieve 100% code coverage with the help of AI? Because code coverage is a really important topic to be used whenever we are evaluating any type of code base, whenever, let's say, we are creating a new build or we are testing out new deployments, new changes that have been made. If you're able to achieve even 90% or 95% code coverage, that itself is considered to be a really great point. But how can we achieve close to 100% code coverage with the help of AI? And that is something that AI can actually help with, because of the fact that we are able to evaluate the code with the help of AI and we are able to generate automated tests, we are able to run the most important tests that are really important for that particular code change, and that will actually help to just understand all the different nuances of the code that are there, including the code changes and how the new changes have actually impacted your older code. All of that can be considered while trying to do code coverage with the help of AI. And then even within the automation testing framework, how can we actually use AI, or you know, let's say, with respect to Selenium, so that whenever you're doing any kind of an automated testing, AI can actually help improve it by looking specifically at the issues that might arise, and we are like using a manual, we are using basically automation testing to look specifically for those particular issues and try to be able to resolve them. So, those are some of the scenarios today where AI is actually being used in software testing.

QnA

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Network Requests with Cypress
TestJS Summit 2021TestJS Summit 2021
33 min
Network Requests with Cypress
Top Content
Cecilia Martinez, a technical account manager at Cypress, discusses network requests in Cypress and demonstrates commands like cydot request and SCI.INTERCEPT. She also explains dynamic matching and aliasing, network stubbing, and the pros and cons of using real server responses versus stubbing. The talk covers logging request responses, testing front-end and backend API, handling list length and DOM traversal, lazy loading, and provides resources for beginners to learn Cypress.
Testing Pyramid Makes Little Sense, What We Can Use Instead
TestJS Summit 2021TestJS Summit 2021
38 min
Testing Pyramid Makes Little Sense, What We Can Use Instead
Top Content
Featured Video
Gleb Bahmutov
Roman Sandler
2 authors
The testing pyramid - the canonical shape of tests that defined what types of tests we need to write to make sure the app works - is ... obsolete. In this presentation, Roman Sandler and Gleb Bahmutov argue what the testing shape works better for today's web applications.
Full-Circle Testing With Cypress
TestJS Summit 2022TestJS Summit 2022
27 min
Full-Circle Testing With Cypress
Top Content
Cypress is a powerful tool for end-to-end testing and API testing. It provides instant feedback on test errors and allows tests to be run inside the browser. Cypress enables testing at both the application and network layers, making it easier to reach different edge cases. With features like AppActions and component testing, Cypress allows for comprehensive testing of individual components and the entire application. Join the workshops to learn more about full circle testing with Cypress.
Test Effective Development
TestJS Summit 2021TestJS Summit 2021
31 min
Test Effective Development
Top Content
This Talk introduces Test Effective Development, a new approach to testing that aims to make companies more cost-effective. The speaker shares their personal journey of improving code quality and reducing bugs through smarter testing strategies. They discuss the importance of finding a balance between testing confidence and efficiency and introduce the concepts of isolated and integrated testing. The speaker also suggests different testing strategies based on the size of the application and emphasizes the need to choose cost-effective testing approaches based on the specific project requirements.
Playwright Test Runner
TestJS Summit 2021TestJS Summit 2021
25 min
Playwright Test Runner
Top Content
The Playwright Test Runner is a cross-browser web testing framework that allows you to write tests using just a few lines of code. It supports features like parallel test execution, device emulation, and different reporters for customized output. Code-Gen is a new feature that generates code to interact with web pages. Playwright Tracing provides a powerful tool for debugging and analyzing test actions, with the ability to explore trace files using TraceViewer. Overall, Playwright Test offers installation, test authoring, debugging, and post-mortem debugging capabilities.
Everyone Can Easily Write Tests
TestJS Summit 2023TestJS Summit 2023
21 min
Everyone Can Easily Write Tests
Playwright is a reliable end-to-end testing tool for modern web apps that provides one API, full isolation, fast execution, and supports multiple languages. It offers features like auto-weighting, retrying assertions, seamless testing of iframes and shadow DOM, test isolation, parallelism, and scalability. Playwright provides tools like VS Code extension, UiMode, and Trace Viewer for writing, debugging, and running tests. Effective tests prioritize user-facing attributes, use playwright locators and assertions, and avoid testing third-party dependencies. Playwright simplifies testing by generating tests, providing code generation and UI mode, and allows for easy running and debugging of tests. It helps in fixing failed tests and analyzing DOM changes, fixing locator mismatches, and scaling tests. Playwright is open source, free, and continuously growing.

Workshops on related topic

Designing Effective Tests With React Testing Library
React Summit 2023React Summit 2023
151 min
Designing Effective Tests With React Testing Library
Top Content
Featured Workshop
Josh Justice
Josh Justice
React Testing Library is a great framework for React component tests because there are a lot of questions it answers for you, so you don’t need to worry about those questions. But that doesn’t mean testing is easy. There are still a lot of questions you have to figure out for yourself: How many component tests should you write vs end-to-end tests or lower-level unit tests? How can you test a certain line of code that is tricky to test? And what in the world are you supposed to do about that persistent act() warning?
In this three-hour workshop we’ll introduce React Testing Library along with a mental model for how to think about designing your component tests. This mental model will help you see how to test each bit of logic, whether or not to mock dependencies, and will help improve the design of your components. You’ll walk away with the tools, techniques, and principles you need to implement low-cost, high-value component tests.
Table of contents- The different kinds of React application tests, and where component tests fit in- A mental model for thinking about the inputs and outputs of the components you test- Options for selecting DOM elements to verify and interact with them- The value of mocks and why they shouldn’t be avoided- The challenges with asynchrony in RTL tests and how to handle them
Prerequisites- Familiarity with building applications with React- Basic experience writing automated tests with Jest or another unit testing framework- You do not need any experience with React Testing Library- Machine setup: Node LTS, Yarn
How to Start With Cypress
TestJS Summit 2022TestJS Summit 2022
146 min
How to Start With Cypress
Featured WorkshopFree
Filip Hric
Filip Hric
The web has evolved. Finally, testing has also. Cypress is a modern testing tool that answers the testing needs of modern web applications. It has been gaining a lot of traction in the last couple of years, gaining worldwide popularity. If you have been waiting to learn Cypress, wait no more! Filip Hric will guide you through the first steps on how to start using Cypress and set up a project on your own. The good news is, learning Cypress is incredibly easy. You'll write your first test in no time, and then you'll discover how to write a full end-to-end test for a modern web application. You'll learn the core concepts like retry-ability. Discover how to work and interact with your application and learn how to combine API and UI tests. Throughout this whole workshop, we will write code and do practical exercises. You will leave with a hands-on experience that you can translate to your own project.
Leveraging LLMs to Build Intuitive AI Experiences With JavaScript
JSNation 2024JSNation 2024
108 min
Leveraging LLMs to Build Intuitive AI Experiences With JavaScript
Featured Workshop
Roy Derks
Shivay Lamba
2 authors
Today every developer is using LLMs in different forms and shapes, from ChatGPT to code assistants like GitHub CoPilot. Following this, lots of products have introduced embedded AI capabilities, and in this workshop we will make LLMs understandable for web developers. And we'll get into coding your own AI-driven application. No prior experience in working with LLMs or machine learning is needed. Instead, we'll use web technologies such as JavaScript, React which you already know and love while also learning about some new libraries like OpenAI, Transformers.js
Detox 101: How to write stable end-to-end tests for your React Native application
React Summit 2022React Summit 2022
117 min
Detox 101: How to write stable end-to-end tests for your React Native application
Top Content
WorkshopFree
Yevheniia Hlovatska
Yevheniia Hlovatska
Compared to unit testing, end-to-end testing aims to interact with your application just like a real user. And as we all know it can be pretty challenging. Especially when we talk about Mobile applications.
Tests rely on many conditions and are considered to be slow and flaky. On the other hand - end-to-end tests can give the greatest confidence that your app is working. And if done right - can become an amazing tool for boosting developer velocity.
Detox is a gray-box end-to-end testing framework for mobile apps. Developed by Wix to solve the problem of slowness and flakiness and used by React Native itself as its E2E testing tool.
Join me on this workshop to learn how to make your mobile end-to-end tests with Detox rock.
Prerequisites- iOS/Android: MacOS Catalina or newer- Android only: Linux- Install before the workshop
API Testing with Postman Workshop
TestJS Summit 2023TestJS Summit 2023
48 min
API Testing with Postman Workshop
Top Content
WorkshopFree
Pooja Mistry
Pooja Mistry
In the ever-evolving landscape of software development, ensuring the reliability and functionality of APIs has become paramount. "API Testing with Postman" is a comprehensive workshop designed to equip participants with the knowledge and skills needed to excel in API testing using Postman, a powerful tool widely adopted by professionals in the field. This workshop delves into the fundamentals of API testing, progresses to advanced testing techniques, and explores automation, performance testing, and multi-protocol support, providing attendees with a holistic understanding of API testing with Postman.
1. Welcome to Postman- Explaining the Postman User Interface (UI)2. Workspace and Collections Collaboration- Understanding Workspaces and their role in collaboration- Exploring the concept of Collections for organizing and executing API requests3. Introduction to API Testing- Covering the basics of API testing and its significance4. Variable Management- Managing environment, global, and collection variables- Utilizing scripting snippets for dynamic data5. Building Testing Workflows- Creating effective testing workflows for comprehensive testing- Utilizing the Collection Runner for test execution- Introduction to Postbot for automated testing6. Advanced Testing- Contract Testing for ensuring API contracts- Using Mock Servers for effective testing- Maximizing productivity with Collection/Workspace templates- Integration Testing and Regression Testing strategies7. Automation with Postman- Leveraging the Postman CLI for automation- Scheduled Runs for regular testing- Integrating Postman into CI/CD pipelines8. Performance Testing- Demonstrating performance testing capabilities (showing the desktop client)- Synchronizing tests with VS Code for streamlined development9. Exploring Advanced Features - Working with Multiple Protocols: GraphQL, gRPC, and more
Join us for this workshop to unlock the full potential of Postman for API testing, streamline your testing processes, and enhance the quality and reliability of your software. Whether you're a beginner or an experienced tester, this workshop will equip you with the skills needed to excel in API testing with Postman.
Monitoring 101 for React Developers
React Summit US 2023React Summit US 2023
107 min
Monitoring 101 for React Developers
Top Content
WorkshopFree
Lazar Nikolov
Sarah Guthals
2 authors
If finding errors in your frontend project is like searching for a needle in a code haystack, then Sentry error monitoring can be your metal detector. Learn the basics of error monitoring with Sentry. Whether you are running a React, Angular, Vue, or just “vanilla” JavaScript, see how Sentry can help you find the who, what, when and where behind errors in your frontend project. 
Workshop level: Intermediate