Machine Learning based Unit Tesing in JavaScript

Hello everyone. Welcome to my talk at TestJS Summit 2022. My talk title is Machine Learning based unit testing in JavaScript. A very quick introduction about myself. I'm Shabail Amba, a TensorFlow.js working group member and a Google support mentor at TensorFlow.

So first question that of course will come to our mind is why machine learning and software testing. Now, if you want to understand more about how machine learning is actually used in software testing, I'd recommend you to go and watch my last year's video at TestJS Summit 2021 where I presented a talk at predictive testing in JavaScript with the help of machine learning and how you can basically use machine learning for being able to not just predict what type of test cases should be put but also how you can make your test coverage and basically make better test cases with the help of machine learning. So in case you're interested in that, I'll definitely recommend you to check this out. But of course today's session is all about unit testing and how you can basically include machine learning in unit testing.

So in case you're not aware of what unit testing is all about, so basically unit test is a way of testing the smallest pieces of code and these could be let's say a function, a sub-routine or a method that might be defined in a programming language. And of course we have different types of testing techniques like integration testing but unit testing is by far the smallest testing procedure that works on a single function or a single method. And that makes unit testing also a really important part of your entire test driven cycle, because it's very important to ensure that we are able to test each and every smallest functionality of a software because if you have a very large code base, even a small function that might not work properly might end up causing some unknown errors inside of your entire code base. So it's really important, and unit test cases form a very important aspect of the entire test driven programming for any organization where testing is being conducted.

But one of the drawbacks that we can discuss about unit testing is that it's a very manual process. So whenever software testers are writing unit test cases for functions, now let's say if you have a very large code base, you could have a multitude of different functions that you're writing. So if you end up writing unit test cases for each one of them, then it can take a lot of time to just write and it's a lot of manual work. So over the past few years, a lot of effort is going into automating the unit test case procedures. And again, the main idea is that with the help of the automation in unit testing, we are able to actually capable of reducing the boilerplate code that is required for unit test cases. And we are also able to suggest and create useful a combination of test data that can be provided to software testers to be used as part of their unit testing capabilities. And this overall helps us to actually improve our code coverage as well. And helps us to write custom test cases that might have been missed out by the software developers. And that's where the advancements in machine learning and AI is basically helping us to be able to do this automation in unit test casing and helping us to improve the overall test coverage that was more difficult to do manually if there is manual test cases being written by humans. So today, we are going to be taking a look at a few of the tools that are available today to software developers to be able to do automation in unit testing with the help of and specifically within either JavaScript or TypeScript. Now the first tool that we're going to be taking a look at is the Pony Code tool. So Pony Code is an AI powered extension that helps to basically generate unit test cases for you and based on the functions that you write inside of your code base, it will recommend certain test cases for you and automatically generate the unit test file for you and this essentially is also done with the help of a very intuitive graphic interface. So you basically get either, uh, like, you know, you can either use the VS code extension, or you can also use the CLI that is provided by Pony Code that, uh, can be used for let's say writing unit test cases or automating the unit test cases for an entire, uh, directory. And you can also use, uh, the CLI tool, uh, as mentioned, like the CLI tool as well. Uh, and you can also use GitHub actions, uh, provided by Pony Code that allow you to basically combine test generation during your CI-CD platform. So let's basically take a look at the product demo. And of course, if you want to explore, uh, deeper into how to get started Pony Code, I'll recommend you to go ahead and take a look at docs.ponycode.com.

But for, uh, this demo, we'll just quickly now move on to VS code. And first let's basically go to this particular link for the docs. So as you can see, like you can either install the Pony code VS code extension or the CLI or you can also use it as a GitHub action, and it's free, free to use for individual developers.

So if you click on the VS code extension and install it inside of your VS code, it will look something like this. So if you're basically using, uh, you can see that it will basically, uh, show up over here as this Pony Code icon inside of your VS code. And you can take a look at some of the things that has the getting started page. Uh, and this basically shows you how to very quickly get started and generate a unit test cases inside of your application.

So, uh, the one that we're going to be using is the Pony Code VS code extension. So in this case, uh, in my file directory, I have created a Pony Code, um, folder. And over here, what I've done is I've just created a simple index.cs file, and I have just created a simple function for adding two numbers. So first let's take a look how this will basically So, um, as soon as you basically write any function, uh, Pony Code will automatically be able to detect the function that you have written, like a JavaScript function. And over here, you can directly see that, uh, we get this, uh, UI graphical user interface where you can see that we can click on this, uh, Pony Code unit test, which will basically generate a UI, um, platform. And you can see that it basically comes up with some suggestions for unit test cases that you can basically write.

So we, you can see that these are the suggestions being provided by Pony Code in order for you to, let's say, increase your test coverage. So let's say that, uh, I want to probably take this example of A and B. Basically, since it's an add to numbers function, we are having two variables and here it takes these two values. So what I can very easily do is that either I can test this particular function. And see that, you know, uh, it basically just generates the response and I can actually click on the plus sign. If I want to basically add this particular specific test case inside of my, uh, Pony Code. And what it immediately does is that now we have a new folder in place and this folder is the Pony Code folder. So if I go and explore this Pony Code folder, you can see that it is the index.ts.js file. So this is an automated, automatically created, uh, test generation file that has been created by Pony Code. So if we basically take a look at, like, you know, what it includes, right. So you can see that, uh, it has, first of all, what it does, it has done. Is that it has, uh, like, you know, imported our index.js file and now it has basically described a function, uh, index.addToNumbers because that's a function that we like, you know, have from our index.js file that we have created and we have generated a over here where it is, uh, like, you know, getting to these two values. Now, let's say if we want to add some additional test cases, we can very easily just click on the UI, uh, on the Pony Code UI and add that. So that will basically automatically get added, uh, inside of your index.test.js function. As you can see that we have another, uh, like, you know, function, another function that we have added, another test case that we have added. So this way you can either remove or delete, uh, these test cases and automatically the unit test that is being generated for you, uh, will be changed. Now let's also take an example for an async function.

So let's define an async function, right? Uh, because Pony Code actually works with both static functions and async functions as well. So let's say we define an async function as let's say get, uh, first name. And let's say it is for a person and over here, we'll go ahead and define this. So let's say that what we do is we return, uh, basically person.name. So if we actually click on the Pony Code unit test generation, it will automatically like, you know, be able to actually understand that, uh, this is basically a, a way based function, and you can see that by giving a proper function name it is very intelligently able to also understand that what we are using is basically names.

So it basically gives suggestions for like, you know, the names as name let's say as a peer.org here, and you can like, you know, take some examples. So if I again click on this plus icon, uh, we should again see this inside of our function. So as you can see that it basically describes the same, the function, and you can see that it automatically detects that it's an async function. So it automatically provides us with an async of it, uh, inside of the test case that we have basically written. So you can see that it very intelligently is able to understand what functions that you're writing and automatically suggest some test cases for you. And if you add those test cases, it will generate the file and it will automatically generate the unit test case for you. So you don't have to write it on your own. So this is really great. If you are let's say, just getting started with software testing with unit testing, right. Or let's say if you're willing to actually increase your test coverage. So you can add as new number of test cases that you want with the help of these suggestions. So that is why like the Pony Code Toolkit is very easy to use and very strong. And it's supported across multiple languages as well, JavaScript, TypeScript and even Python, Java. So you can also use for other languages as well, if you want. So it's as easy to use. And of course, if you want for it to be part of your CICD, then you can basically include it as part of your CICD workflow as well.

Now, so that is all about a very quick introduction about how basically Pony Code works. So in case you want to install it, you can very easily just go ahead and install the VS Code extension or the CLI platform. Now, we'll basically move ahead with our next one. So the next tool that we'll take a look at is the GitHub Copilot. Now, what exactly is GitHub Copilot? It's basically an AI pair programmer that was created using the OpenAI Codex. What it essentially does is that it basically looks out for all different types of public code bases, including all the open source repositories on GitHub. And the OpenAI Codex model basically inputs all of this entire code that is provided. And it is able to, of course, with the help of AI, is able to recommend new functions whenever you're writing code. So, it acts as an auto-completion, but of course as an intelligent code auto-completion.

And it can suggest you complete lines of code or entire functions as well by just defining these function names. And today we will see how we can basically use GitHub Copilot in the scenario of actually, let's say, auto-filling the entire unit cases. Now, of course, in comparison to the Pony code, it cannot just suggest or it does not provide you a UI specifically because the GitHub Copilot is not like a dedicated software testing automation tool. It is essentially an auto-completion code. But what GitHub Copilot can do is it can understand the context of your entire directory or of your code that you're writing and then appropriately auto-fill your code that you as you write. So let's say, if you're writing some unit test cases manually, it can at least give you suggestions for auto-completion of your unit test cases that you're writing. And we will take a look at a quick demonstration of how that basically works.

So let's take a look at basically the demo for the GitHub Copilot. So what we'll do is we'll basically go ahead and take a look at. So I've basically, within my VS code, I have created another, like another directory over here, which is with the name of Copilot. And over here, I have an index.js function. And I'll just close out all the other ones that are not required. And let's go ahead and save it. Let's just close all of this. Yeah, so this is the very simple function that I've created over here. I have English code and Spanish code. And then what I've done is like, we have this as, we have a simple function about get about us link. So basically based on which particular, like which particular code that we provide, whether it is like, you know, in English or it is in Spanish. So it will return the about us page, like, you know, based on that specific language. So what we'll go ahead and do is that we'll basically create a unit test case for this.

Now, of course, as a dev dependency, something to keep in mind is that we are going to be installing, we are going to be using zests as our dependency. So over here, you can see that what we are going to be using is that we are, we are basically using zests. So in case you are like, you know, also being to use, we can very easily just like, you know, do yarn add zests and that will basically add like, you know, the zests NPM package for you. So just install, this is basically used for like, no JavaScript testing. So you can install the specific, zests based NPM package, and I'll quickly navigate now to the CoPilot folder. So let me do that. Now we are inside of CoPilot. So what we'll do is that, of course, with zests, the way like, you know, you write test cases that you create a, like you know, in basically a underscore dot test.js file. In this case, like, you know, whatever is your main file, you'll create a corresponding file over there. So this is where we basically write a test case.

Now, in order to basically get started with GitHub CoPilot, we can visit, like another GitHub CoPilot. And, like, you know, this is basically the GitHub CoPilot. So, again, what, how it basically works is that you can install it as an extension on your, on your VS code. So, of course, right now it is paid. But if you, let's say, if you are an open source maintainer, then you can basically get GitHub CoPilot for free. So if you, if you are an open source maintainer or contributor, you can get it for free as well. But if you're a student as well, so you can definitely get it for free. So, again, how it basically looks is that once you have successfully installed it on your GitHub, then you can on your VS code, you will find the CoPilot icon over here. So you can see that right now it is enabled for me. And you can also enable it again in a few clicks, you'll be able to enable it and use it.

So now what I'll go ahead and do is that as like, you know, you'll see that as you type something, it will auto suggest code for you. So let's say that I write and we have a comment that let's say, write a unit test case. So it will basically understand, like, you know, the context. And you can see like, it is already providing something like a solution for the function auto to numbers. But of course, what we'll just do it and we'll like, you know, we'll type let's say for, we can say like, let's say, write a unit test case for index.js file, right? So we are basically going ahead and going to be using that. So let's go ahead and actually start writing our function.

So as you can see, like, you know, it automatically knows the path. So let's take this as like a sufficient from the Github copilot. And now let's go ahead and write our actual function. So like, you know, first, what we'll do is that we'll first get let's say, we'll define a, like, you know, we'll define a variable get about us link. And this will basically require from our index function that we have created. So we heard, like, you know, we can get it from the, from the index, right? So we'll get it from our index. Now let's define our test. So in this case, the test that we'll define. So let's define our test over here. Let's say that we define it as, like, you know, in this case, let me right. Okay. So let's basically define our test as returns about us in English language. Let's say in English language, right. So as you can see, like, as you're typing, it is automatically providing us like, you know, suggestions for So in this case, I'll just complete this code over here.

And now we'll basically write our code. So we'll see that we expect, right. So let's define this. And as you can see that it automatically suggests us the code that, like, you know, we are basically using the get about this link, and we are going to be using the English function and it should be, like, you know, about us. So let's just complete this. And let's use this and let's go ahead and save this.

So just like, you know, this particular function and use just so as you can see, okay, so it probably failed for a second, let's just go ahead and ensure that it works over here. So we'll quickly take a look at the index.js file and it should be about us. So let's actually look at the test and the test. Let's just try to see if we are able to run this. So we'll use test and as you can see that it basically passes, right? So that's the idea that you know with the help of the GitHub Copilot. Although it does not provide you an entire UI something very similar to the Bonnie code, but it still is intelligent enough to understand the context of your entire directory and then provide you auto-suggestions for code as you write. So you can basically very easily write your test cases with the help of the auto suggestions that are provided by the GitHub Copilot.

So of course these were a couple of the different tools that we basically took a look at. Now there are a lot of other different tools as well that today are being created and that are available for other languages as well, but these were a couple of the demonstrations that I wanted to showcase for JavaScript. So in order to summarize, so whether you want to basically get started with unit test cases or if you want to increase your entire code coverage instead of your test driven programming, tools like PonyCode and GitHub Copilot allow you to basically do that and allow you to decrease that manual effort that is required to be able to generate so many test cases and it also helps you to think about those kind of test cases that you might have probably not covered or might have forgotten to add and that's why these tools are very powerful for helping improve the overall code coverage and helping achieve that dream of achieving 100% code coverage and especially in this area of today's generation of test driven programming, the adoption of these tools is going to be improving a lot and automation of unit test case is definitely on the way ahead. With that, thank you so much for attending my talk and you can connect with me on my Twitter at the RedHotDevelop and on my GitHub as well and in case you have any questions, feel free to reach out and I'll be also hanging out in TestJS Summit. So, if you want to connect with me, I'll be more than happy to let you know talk about AI and how AI is revolutionizing software testing. Thank you so much to the entire team and I hope to see you next year in TestJS Summit 2023.